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The  Honorable  James  Inhofe 
Chairman 

The  Honorable  Charles  Robb 
Ranking  Minority  Member 

Subcommittee  on  Readiness  and  Management  Support 
Committee  on  Armed  Services 
United  States  Senate 

As  you  requested,  this  report  assesses  the  extent  the  Department  of  Defense’s  (DOD)  training  helps 
weapon  system  program  offices  apply  best  practices  and  whether  such  training  can  be  of  more  help. 
We  make  recommendations  to  the  Secretary  of  Defense  on  how  DOD  training  can  better  support 
program  offices  in  the  application  of  best  practices. 

We  are  sending  copies  of  this  report  to  the  Honorable  William  S.  Cohen,  Secretary  of  Defense;  the 
Honorable  Louis  Caldera,  Secretary  of  the  Army;  the  Honorable  Richard  Danzig,  Secretary  of  the 
Navy;  the  Honorable  F.  Whitten  Peters,  Acting  Secretary  of  the  Air  Force;  the  Honorable  Jacob  J.  Lew, 
Director,  Office  of  Management  and  Budget;  and  to  interested  congressional  committees.  We  will  also 
make  copies  available  to  others  upon  request. 


If  you  or  your  staff  have  any  questions  regarding  this  report,  please  call  me  at  (202)  512-4841. 
Contacts  and  key  contributors  to  this  report  are  listed  in  appendix  in. 


Katherine  V.  Schinasi 
Associate  Director 
Defense  Acquisitions  Issues 


Executive  Summary 


Purpose 


The  Department  of  Defense  (DOD)  plans  to  increase  its  annual 
procurement  investment  to  about  $60  billion  by  fiscal  year  2001.  DOD  has 
high  expectations  from  this  investment:  that  new  weapons  will  be  better 
yet  less  expensive  than  their  predecessors  and  will  be  developed  in  half  the 
time.  Essential  to  getting  these  kinds  of  outcomes  will  be  the  adaptation  of 
best  commercial  practices  that  have  enabled  leading  commercial  firms  to 
develop  new  products  faster,  cheaper,  and  better.  DOD  has  begun  a  number 
of  acquisition  reform  initiatives  based  on  commercial  practices  to  help 
foster  these  outcomes.  Their  success  depends  greatly  on  the  extent  to 
which  the  program  offices  responsible  for  managing  weapon  acquisitions 
can  implement  the  practices  on  individual  programs.  Training  provided  to 
the  program  offices  serves  as  a  key  agent  in  both  creating  a  culture  that  is 
receptive  to  new  practices  and  in  providing  the  knowledge  needed  to 
implement  new  practices  at  the  workplace.  At  the  request  of  the  Chairman 
and  the  Ranking  Minority  Member,  Subcommittee  on  Readiness  and 
Management  Support,  Senate  Committee  on  Armed  Services,  GAO 
evaluated  the  role  DOD  training  is  playing  in  implementing  best  practices  in 
weapon  system  programs.  This  report  addresses  (1)  the  contribution  DOD 
training  makes  to  program  offices’  ability  to  apply  best  practices,  (2)  the 
different  methods  used  by  DOD  and  leading  commercial  firms  in  training 
on  best  practices,  and  (3)  the  strategic  approaches  that  underlie  DOD’s  and 
leading  commercial  firms’  training  methods  for  best  practices. 


Background 


GAO’s  review  focused  on  weapon  system  program  offices  because  they 
comprise  a  key  component  of  DOD’s  acquisition  workforce.  In  planning, 
managing,  and  executing  acquisition  programs,  these  program  offices  are 
responsible  for  managing  about  $80  billion  of  DOD’s  annual  research, 
development,  and  procurement  funds.  As  an  entry  point  for  DOD 
acquisitions,  program  offices  greatly  influence  the  work  of  the  rest  of  the 
acquisition  workforce.  The  primary  responsibility  for  training  the 
acquisition  workforce  falls  within  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology.  In  1992,  the  Defense  Acquisition 
University,  a  consortium  of  13  schools,  was  created  to  develop  and  provide 
training  for  the  acquisition  workforce.  Each  service  also  has  an  acquisition 
reform  office  that  provides  the  workforce  with  the  latest  information  about 
practices  and  initiatives  that  apply  to  acquisitions.  Based  on  personnel 
reductions  mandated  by  the  National  Defense  Authorization  Acts  for  Fiscal 
Years  1996  and  1997,  DOD  expects  the  acquisition  workforce  of  fiscal  year 
2000  to  be  25  percent  smaller  than  that  of  fiscal  year  1995.  Thus,  training 
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will  become  even  more  important  as  new  authority  and  responsibility  is 
granted  to  those  who  remain  in  the  workforce. 

To  determine  the  extent  to  which  DOD  training  and  other  sources  helped 
program  offices  obtain  the  knowledge  needed  for  implementing  best 
practices,  GAO  focused  on  five  specific  practices:  cost  as  an  independent 
variable,  integrated  product  teams,  evaluation  of  contractors’  past 
performance,  setting  performance  specifications,  and  managing  supplier 
relationships.  The  first  four  are  formal  DOD  initiatives  that  are  based  on 
best  commercial  practices,  while  supplier  relationships  is  a  best  practice 
GAO  has  observed  in  leading  commercial  firms.  GAO  selected  six  program 
offices  considered  by  DOD  as  leaders  in  implementing  one  or  more  of  the 
practices.  As  such,  they  represented  best  case  examples  of  marshaling 
training  and  other  resources  needed  to  implement  new  acquisition 
practices.  The  term  “standard  training”  is  used  in  this  report  to  describe 
the  training  provided  by  the  Defense  Acquisition  University  and  the 
services’  acquisition  reform  offices,  as  distinct  from  training  that  program 
offices  provide  on  their  own. 


Results  in  Brief  DOD’s  standard  training  did  not  make  a  major  contribution  to  the  leading 

program  offices’  ability  to  implement  best  practices.  In  evaluating  their  key 
sources  of  knowledge,  none  of  the  key  officials  from  programs  at  the 
forefront  of  implementing  best  practices  ranked  standard  DOD  training 
first,  with  many  ranking  it  last.  DOD  training  either  did  not  reach  the  right 
people  when  it  was  needed  or  did  not  reach  them  at  all.  When  training  on 
best  practices  was  received,  it  did  not  contain  the  depth  or  practical 
insights  program  office  people  needed  to  implement  the  practices.  It  was 
primarily  through  their  own  efforts — Gleaming  on  the  job,  finding  external 
training,  or  developing  their  own  training  program — ^that  they  attained  the 
knowledge  needed  to  apply  best  practices.  'Thus,  success  depended  on 
their  having  the  foresight  to  see  what  was  needed,  the  ingenuity  to  find 
good  sources  of  knowledge,  and  the  resources  needed  to  attain  that 
knowledge.  Replicating  this  approach  broadly  on  other  programs  is 
problematic.  Managers  may  not  see  the  relevance  of  a  practice  to  their 
programs  and  thus  may  not  realize  what  training  is  needed.  Others  may  not 
be  able  to  afford  the  needed  training. 

Leading  commercial  firms  and  DOD  use  different  training  methods  to 
implement  key  practices.  Conunercial  firms  use  targeted,  hands-on 
methods  to  ensure  that  program  offices  are  trained  on  key  practices.  Their 
training  organizations  conduct  front-end  analyses  to  determine  the 
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programs’  training  requirements  and  involve  the  program  offices  in 
designing  the  training.  Training  is  customized  to  meet  the  specific  needs  of 
those  implementing  the  practice.  Company  officials  beUeve  the  targeted 
method  results  in  more  useful  training,  which  helps  to  improve  outcomes 
of  the  final  product.  DOD  does  not  have  a  counterpart  to  this  method. 
DOD  relies  primarily  on  its  standard  training,  including  classroom  courses, 
videos,  internet-based  training,  satellite  broadcasts,  and  roadshows,  to 
inform  staff  on  best  practices.  These  methods  were  designed  for  functional 
training,  such  as  for  engineers,  and  for  increasing  the  awareness  of  new 
practices.  As  such,  they  do  not  provide  the  depth  or  reach  enough  of  the 
right  people  at  the  right  time  to  be  of  great  help  in  implementing  best 
practices  at  program  offices.  Also,  DOD  does  not  systematically  involve 
program  offices  in  the  design  of  training. 

The  intensive  training  methods  leading  commercial  firms  employ  on  new 
practices  are  the  result  of  a  strategic,  institutionally  driven  approach  to 
implementation.  These  firms  commit  their  resources  and  attention  to  a  few 
weU-defined  practices  and  make  a  significant  front-end  investment  in  the 
training  to  be  provided  to  the  workforce.  Also,  the  firms  strive  to  create  an 
environment  to  put  those  responsible  for  implementing  the  practices  in  a 
good  position  to  succeed.  DOD’s  training  methods  for  best  practices  do  not 
stem  from  such  a  strategic  approach.  DOD  has  promulgated  as  many  as  40 
acquisition  management  initiatives  in  the  past  few  years  without 
communicating  their  relative  priority  to  trainers  or  implementers.  Often, 
the  initiatives  have  not  been  accompanied  by  clear  guidance  or  by  the 
initial  training  needed  for  implementation.  While  DOD  commits  significant 
resources  to  training,  it  does  not  make  a  uniform  front-end  investment  to 
ensure  that  program  offices  will  succeed  with  the  new  practices.  Since 
1997,  two  studies  commissioned  by  DOD  have  recommended  ways  to  make 
training  organizations  more  effective  in  providing  training  of  best  practices. 
These  recommendations  were  not  adopted  in  favor  of  a  more  traditional 
training  role.  In  June  1999,  another  DOD  study  proposed  that  DOD  training 
organizations  become  change  agents  and  be  modeled  after  their  corporate 
counterparts.  If  the  latest  study’s  recommendations  are  adopted,  DOD  may 
be  in  a  much  better  position  to  provide  the  type  of  help  program  offices 
need  to  successfully  implement  best  practices. 

GAO  makes  recommendations  to  the  Secretary  of  Defense  on  how  DOD’s 
training  on  best  practices  can  better  support  the  needs  of  program  offices. 
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Principal  Findings 


DOD  Training  Not  a  M^or  Training  often  did  not  exist  or  was  not  provided  when  program  officials 

Catalvst  for  Best  Practices  began  to  implement  an  individual  practice.  For  example,  when  the  Joint 

^  Strike  Fighter  program  office  started  to  implement  cost  as  an  independent 

variable,  there  were  no  guiding  documents,  and  no  one,  including  the 
training  community,  knew  what  the  initiative  really  meant.  Some  people 
involved  with  implementing  best  practices  were  missed  altogether  by  DOD 
training  offerings  because  they  had  not  been  required  to  take  training  since 
before  the  initiatives  began.  Others,  such  as  requirements  authors,  fell 
outside  the  definition — and  training  curriculum — of  the  acquisition 
workforce.  In  one  case,  program  officials  reduced  contractor  reporting 
requirements  to  2  items,  in  line  with  acquisition  reform,  only  to  have  40 
more  added  by  another  office  with  approval  authority  over  the  contract. 
Contractors  are  also  essential  to  the  application  of  best  practices,  but  they 
are  not  part  of  the  DOD  defined  acquisition  workforce  or  the  training 
offered.  Consequently,  they  do  not  necessarily  understand  or  know  how  to 
implement  new  practices.  Program  officials  noted  that  DOD’s  standard 
training  typically  provided  a  general  awareness  of  the  practices  but  not  tlie 
“how-to”  knowledge  needed  for  implementation.  Training  was  not  tailored 
in  such  a  way  that  individuals  could  see  how  the  practices  could  be  applied 
to  their  program.  Program  officials  also  observed  that  in  some  cases 
training  suffered  because  instructors  did  not  use  up-to-date  case  studies 
and  were  not  current  on  new  practices  themselves.  They  also  believed  they 
had  little  opportunity  to  influence  the  training  they  received  from  DOD. 

Program  officials  used  a  combination  of  ways — generally  outside  of 
standard  DOD  training  offerings — to  get  the  knowledge  they  needed  to 
apply  best  practices.  Several  program  officials  relied  on  the  cumulative  job 
expertise  of  the  staff  and  personal  research  to  teach  themselves  how  to 
implement  new  initiatives.  For  example,  on  the  Advanced  Medium-Range 
Air-to-Air  Missile  program,  officials  relied  on  their  personal  experiences  to 
know  how  to  set  contract  specifications  at  a  performance,  rather  than  a 
detailed,  level.  Officials  also  used  their  program  funds  to  send  staff  to 
nongovernment  sources  or  to  bring  experts  in.  One  program  manager 
sends  people  to  outside  training,  such  as  imiversity  leadership  courses,  to 
develop  creative  thinking  skills.  One  of  his  managers  said  some  of  his  best 
training  was  from  off-site  sessions  sponsored  by  the  program  office  that 
dealt  with  the  people  issues  critical  to  making  integrated  product  teams 
work.  For  the  Advanced  Amphibious  Assault  Vehicle  program,  the  prime 
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DOD’s  Training  Methods  for 
Best  Practices  Do  Not  Go  as 
Far  as  Leading  Commercial 
Firms’ 


contractor  was  responsible  for  providing  the  training  on  integrated  product 
teams  and  cost  as  an  independent  variable.  The  contractor  hired  a  third 
party  to  develop  a  training  program  tailored  to  the  Advanced  Amphibious 
Assault  Vehicle  program,  and  both  program  office  and  contractor  staff 
were  taught  together,  on-site  as  a  team. 


For  routine  training,  such  as  skill  building,  leading  commercial  firms  have 
standard  training  offerings,  including  functional  area  courses  and 
instruction  on  corporatewide  issues,  such  as  communications  or  ethics. 
However,  when  implementing  key  practices — such  as  those  that  change 
how  product  development  and  production  are  conducted — leading  firms  go 
beyond  standard  training  offerings.  Commercial  firms  use  a  targeted, 
hands-on  approach  to  ensure  program  teams  are  put  in  a  good  position  to 
implement  a  new  practice.  The  elements  common  to  how  leading  firms 
provide  training  on  a  key  practice  include  (1)  front-end  analysis  of  program 
teams’  needs  and  training  requirements,  (2)  involvement  of  program  teams 
in  key  training  decisions,  (3)  customized  training  to  meet  program  team 
specific  needs,  and  (4)  targeted  training  for  the  implementation  of  specific 
practices.  Program  staff  participate  in  and  often  influence  a  wide  range  of 
training  decisions,  including  the  amount  of  training  provided  for  certain  job 
descriptions,  course  topics,  depth  of  course  coverage,  and  identification  of 
the  appropriate  course  recipients.  The  involvement  of  the  program  staff 
has  improved  course  depth,  timeliness,  and  coverage  of  personnel  in  the 
commercial  firms. 

In  the  Boeing  Company,  training  representatives  develop  a  partnership  with 
the  program  staff  when  a  team  is  formulated  to  design  and  manufacture  a 
new  airplane.  The  training  organization  forms  “drop  teams”  to  colocate 
with  the  program  staff  to  learn  as  much  as  possible  about  the  business 
process  and  the  staffs  concerns  and  to  determine  what  training  is  needed 
to  help  the  program  staff  implement  a  practice.  Boeing  officials  stated  that 
training  was  instrumental  to  the  implementation  of  key  practices,  such  as 
design  build  teams,  on  the  777  aircraft  program.  They  noted  that  such 
teams  were  at  odds  with  the  company’s  culture  because  employees  were 
not  accustomed  to  working  in  a  team  environment  and  sharing  information 
across  fimctional  areas.  Training  officials  worked  side  by  side  with  the 
program  staff  to  create  a  training  program  that  provided  team  building  and 
conflict  resolution  techniques  as  well  as  new  technical  skills  training.  To 
ensure  all  program  staff  were  equally  trained,  employees  were  required  to 
complete  training  before  they  reported  for  duty  on  the  program.  The 
professional  employees — engineers  and  drafters — were  required  to 
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complete  120  hours  of  start-up  training  on  several  key  111  practices  before 
they  were  allowed  to  report  for  duty.  Teams  were  often  trained  together  at 
the  work  location.  Ford  followed  a  similar  approach  when  it  implemented 
the  Ford  Product  Development  System — a  lean  engineering  process. 

Responsibility  for  training  on  best  practices  is  diffused  among  several  DOD 
organizations,  including  the  Defense  Acquisition  University  and  the  service 
acquisition  reform  offices.  However,  GAO  did  not  find  an  organization  that 
was  able  to  tailor  and  help  deliver  training  on  best  practices  to  the  program 
offices  visited.  Training  provided  by  the  university  is  designed  primarily  to 
enable  people  in  individual  career  fields  or  functions,  such  as  engineering 
and  cost  estimating,  to  meet  professional  certification  requirements.  The 
university  incorporates  best  practices  topics  into  these  functionally 
oriented  courses  as  drop-in  modules  that  provide  a  survey  of  the  topic,  but 
not  in-depth  coverage.  Although  program  offices  see  a  greater  need  for 
training  that  cuts  across  functions  to  implement  new  practices  and  to 
manage  in  a  team  environment,  it  is  difficult  for  a  person  in  one  career  field 
to  obtain  training  in  another  field.  The  usefulness  of  these  courses  for  best 
practices  is  further  hampered  by  limited  availability;  according  to  an 
official  from  the  university,  the  member  schools  get  about  10  percent  of  the 
workforce  into  training  each  year. 

DOD’s  Acquisition  Reform  Communication  Center  and  the  acquisition 
reform  offices  in  the  services  communicate  best  practice  information 
through  videos,  periodic  satellite  broadcasts,  roadshows,  and  Acquisition 
Reform  Week.  These  methods  can  reach  more  people  than  Defense 
Acquisition  University  courses  and  are  designed  around  practices — ^versus 
functions — ^but  are  not  tailored  to  specific  program  offices  and  are  not 
necessarily  delivered  at  the  time  those  implementing  new  practices  or 
initiatives  need  them.  For  example,  roadshows,  traveling  multiday  training 
workshops  provided  to  staff  at  a  number  of  locations,  typically  provide 
awareness  training  on  the  practices.  DOD  officials  estimate  that  only  10  to 
15  percent  of  the  acquisition  workforce  attend  the  second  day  of 
workshops,  where  more  detailed  training  is  provided.  The  annual 
Acquisition  Reform  Weeks,  which  are  a  combination  of  satellite  broadcasts 
and  local  presentations,  mainly  provide  awareness-level  training.  Neither 
individual  attendance  nor  the  level  of  learning  attained  by  attendees  is 
tracked  by  these  methods. 
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Differences  Between  DOD 
and  Commercial  Training 
Reflect  Different  Strategic 
Approaches 


Leading  commercial  firms  shared  a  common  strategy  for  adopting  and 
deploying  key  new  practices.  First,  the  firms’  corporate  management 
committed  to  and  adopted  few  key  practices — seven  or  less — at  any  given 
time.  In  doing  so,  the  companies  were  able  to  concentrate  their  attention 
and  target  resources  to  implementing  the  practices.  It  also  signaled  the 
importance  of  the  practices  to  trainers  and  implementers.  Second,  the 
firms  assessed  which  staff  should  be  included  in  the  implementation.  For 
example.  Ford’s  training  unit  determined  which  engineering  teams  working 
on  vehicle  lines  could  benefit  from  the  new  production  development 
system.  Third,  company  leaders  made  implementing  the  practice 
mandatory  for  the  target  population.  Lastly,  companies  developed 
well-defined  learning  objectives  to  better  ensure  that  the  target  population 
consistently  understood  how  to  apply  a  new  practice  to  improve 
production  outcomes — the  ultimate  goal.  According  to  officials  from  these 
firms,  it  was  a  corporate  responsibility  to  ensure  that  those  implementing 
the  practice  received  the  necessary  training  and  other  assistance  to 
succeed.  It  was  for  this  reason  that  the  companies  made  a  significant 
front-end  investment  to  support  the  needs  of  program  offices  that  would 
implement  key  practices.  Company  officials  also  pointed  out  that  training 
is  just  one  of  the  Several  components  necessary  for  adopting  new  practices. 
'They  stressed  that  providing  strong  leadership  and  the  right  environment 
were  key  to  ensuring  the  implementation  of  new  practices  and  to 
developing  quality  training. 


DOD’s  approach  to  implementing  best  practices  is  less  structured  and  more 
reliant  on  individual  program  offices  to  make  the  necessary  training 
investment.  DOD  policies  on  individual  reform  initiatives  are  typically 
promulgated  without  indicating  what  components  of  the  acquisition 
workforce  or  which  programs  are  expected  to  implement  the  practice.  In 
addition,  the  policies  thenvselves  are  not  always  clear.  For  example, 
although  the  initiative  on  cost  as  an  independent  variable  was  promulgated 
in  1995,  Advanced  Amphibious  Assault  Vehicle  program  officials 
developed  training  in  1997  to  define  the  initiative  for  the  program.  In 
November  1998,  an  Air  Force  workshop  on  the  cost  initiative  reported  that 
it  was  still  not  well  understood  or  widely  implemented.  DOD  has  proposed 
over  40  reform  initiatives  since  1994,  without  an  indication  of  relative 
priority,  leaving  educators  and  implementers  to  decide  what  is  important. 
Program  offices  are  not  necessarily  in  a  good  position  to  sort  through  the 
initiatives  to  focus  on  those  that  are  the  most  important  to  the  job  at  hand. 
A  service  acquisition  reform  official  observed  that  the  combination  of  many 
reform  initiatives  and  unclear  priorities  causes  the  office  to  guess  what  is 
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the  most  important,  which  leads  to  emphasizing  what  is  perceived  to  be 
popular. 

DOD  is  aware  of  the  need  to  improve  the  means  by  which  the  acquisition 
workforce  receives  and  implements  new  initiatives.  A 1997  study  by  a  DOD 
team  and  a  1998  study  by  the  Logistics  Management  Institute  pointed  out 
several  weaknesses  in  the  focus  and  delivery  of  DOD  training.  Weaknesses 
included  the  need  for  the  Defense  Acquisition  University  to  be  more  active 
in  implementing  best  practices  and  reforms,  the  tendency  for  the  training 
curriculum  to  be  functionally  stove-piped,  and  the  need  for  the  university 
to  have  more  involvement  with  the  workforce — the  recipients  of  training. 
The  studies  made  recommendations  for  significant  organizational  and 
operational  changes  in  the  university,  including  that  it  should  follow  the 
corporate  university  model  of  becoming  a  change  agent  and  a  proponent  of 
best  practices  and  put  more  emphasis  on  cross-functional  training.  In 
September  1998,  the  university  proposed  a  revised  structure,  which  has  not 
been  approved.  While  the  proposal  offers  some  improvements,  it  stays 
close  to  its  roots  of  providing  functional  training.  It  does  not  reflect  the 
corporate  university  model,  a  broadened  role  as  a  change  agent,  or  a  closer 
relationship  with  the  acquisition  workforce.  In  short,  it  does  not 
discemibly  address  key  weaknesses  in  providing  training  of  best  practices. 

In  June  1999,  a  DOD  study  team  chartered  to  identify  training  on 
commercial  business  practices  for  the  acquisition  workforce  concluded 
that  adopting  the  most  effective  commercial  practices  requires  a  cultural 
and  organizational  transformation  within  DOD.  The  team  proposed  a 
cross-functional  plan  for  managing  acquisitions  that  embraces  best 
practices  and  calls  for  “learning  organizations  that  seek  out  and  adopt  best 
practices  that  improve  individual  and  organizational  performance.”  The 
plan  proposes  new  roles  for  several  organizations  in  fostering  the  adoption 
of  best  practices.  Among  these  is  the  Defense  Acquisition  University.  For 
example,  it  recommends  that  the  university  be  broadly  recast  to  adopt  the 
corporate  university  model  and  become  a  change  agent.  This  plan,  while 
not  specific  about  the  help  that  program  offices  would  receive,  does  call  for 
a  strategic  approach  that  would  make  it  more  likely  that  DOD  could 
provide  its  program  offices  tailored  training — more  help — ^in  implementing 
best  practices.  However,  the  fact  that  the  September  1998  and  the  June 
1999  proposals  are  still  vying  for  approval  indicates  that  DOD  has  not  yet 
decided  what  role  it  wants  acquisition  training  to  play  on  best  practices. 
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Recommendations 


Agency  Comments 


GAO  makes  several  recommendations  to  the  Secretary  of  Defense  that  are 
intended  to  ensure  that  DOD’s  approach  to  the  training  of  new  practices 
better  supports  the  needs  of  program  offices  by  (1)  developing  a  strategy 
for  a  structured  approach  to  training  on  new  practices;  (2)  providing 
tailored  training  assistance  to  program  offices;  and  (3)  improving  the 
standard  training  curriculum  so  that  it  is  more  timely,  relevant,  and 
accessible.  These  recommendations  appear  in  full  in  chapter  5. 


DOD  concurred  with  the  views  expressed  in  the  report  and  all  of  the 
recommendations.  A  discussion  of  DOD’s  actions  appears  in  chapter  5. 
DOD’s  comments  appear  in  appendix  I. 
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Introduction 


The  Department  of  Defense  (DOD)  plans  to  increase  its  annual 
procurement  investment  to  $60  billion  by  fiscal  year  2001.  DOD  has  high 
expectations  from  this  investment:  that  new  weapons  will  be  better  and 
less  expensive  than  their  predecessors  and  will  be  developed  in  half  the 
time.  Essential  to  getting  these  kinds  of  outcomes  will  be  the  adaptation  of 
best  commercial  practices  that  have  enabled  leading  commercial  firms  to 
develop  new  products  faster,  cheaper,  and  better.  To  help  foster  these 
outcomes,  DOD  has  begun  a  number  of  acquisition  reform  initiatives  based 
on  commercial  practices.  Success  depends  greatly  on  the  extent  to  which 
the  program  offices  responsible  for  managing  weapon  acquisitions  can 
implement  the  practices  on  individual  programs. 

The  training  DOD  provides  program  office  staff  to  help  them  implement 
best  practices  should  play  a  central  role  in  getting  the  desired  outcomes. 
While  first-hand  experience  and  “learning  by  doing”  are  instrumental  in 
adopting  new  practices,  training  serves  as  a  key  agent  in  creating  a  culture 
that  is  receptive  to  new  practices  and  providing  the  knowledge  to 
implement  the  new  practices  at  the  workplace.  The  relationship  between 
training  and  implementing  new  practices  was  highlighted  in  a  1994  study  of 
300  “improvement-driven”  organizations  conducted  by  Coopers  & 
Lybrand— with  the  American  Society  of  Quality  Control,  Rutgers  University 
Center  for  Public  Productivity,  and  the  National  Institute  of  Canada,  The 
study  found  that  training  played  a  “critical,  integrative  role  as  driver  of 
cultural  change,  process  alignment,  job  redesign,  and  continuous 
improvement.”  Organizations  included  in  the  study  used  training  as  the 
vehicle  for  implementing  and  sustaining  the  changes  at  the  level  where  the 
work  was  done.  The  leading  commercial  firms  we  reviewed  committed 
substantial  investments  to  the  training  on  key  practices,  underscoring  its 
importance  to  getting  the  outcomes  desired  by  the  firms.  The  significant 
numbers  of  reform  initiatives  that  DOD  has  introduced,  which  involve 
adopting  a  number  of  new  practices  in  the  acquisition  of  weapons,  present 
implementation  challenges  that  training  can  help  meet. 


Responsibilities  for 
Training  the 
Acquisition  Workforce 


Within  DOD 


For  nearly  50  years,  the  importance  of  an  educated  professional  DOD 
acquisition  workforce  has  been  emphasized  by  government  leaders  and 
reflected  in  the  work  of  key  studies  and  reform  commissions.  The  First 
and  Second  Hoover  Commissions  (1949  and  1955),  the  Fitzhugh 
Commission  (1970),  and  the  Commission  on  Government  Procurement 
(1972)  all  recognized  the  importance  of  high  quality,  well-educated 
acquisition  professionals  to  the  successful  operation  of  DOD.  The  Packard 
Commission,  which  undertook  a  broad  examination  of  DOD  management 
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practices  and  procedures,  reported  in  its  June  1986  report  that  the  DOD 
acquisition  workforce  was  undertrained  and  inexperienced.  One  of  its 
recommendations  was  to  improve  the  education  and  training  of  the 
acquisition  workforce  for  the  purpose  of  enhancing  the  defense  acquisition 
process. 

Based  in  part  by  the  recommendations  of  the  Packard  Conunission,  the 
Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  was  passed  in 
1990  as  part  of  the  National  Defense  Authorization  Act  for  Fiscal  Year 
1991.^  Its  primary  objective  was  to  improve  the  DOD  acquisition  system  by 
enhancing  the  education,  training,  and  career  development  of  members  of 
the  acquisition  workforce.  Accordingly,  DAWIA  established  the  Defense 
Acquisition  University  (DAU)  to  provide  for  the  professional  educational 
development  and  training  of  the  DOD  acquisition  workforce.  The  act  also 
charged  DOD  officials  with  the  responsibility  to  designate  certain  positions 
as  acquisition  positions,  to  set  qualification  requirements,  and  to  estabhsh 
poUcies  and  procedures  for  training  the  acquisition  workforce. 

The  program  offices  that  manage  weapon  system  acquisitions  are  a  key 
component  of  DOD’s  acquisition  workforce.  This  workforce  is  generally 
defined  as  those  people  who  are  responsible  for  managing  the  wide  array  of 
DOD  acquisitions,  including  contracting  professionals,  program  managers, 
engineers,  scientists,  logisticians,  and  other  occupational  fields,  from  the 
earliest  phases  of  basic  research  to  the  logistical  support  and  disposal  of 
old  systems.  However,  there  have  been  several  definitions  of  what 
comprises  the  DOD  acquisition  workforce,  which  have  led  to  varying 
estimates  of  the  workforce  size,  ranging  from  about  106,000  to  355,000 
people.  At  the  end  of  fiscal  year  1997,  DOD  estimated  the  acquisition 
workforce  covered  by  DAWIA  included  approximately  106,000  positions,  of 
which  about  90,000  were  civilian.  Its  most  recent  definition  places  the 
workforce,  now  called  the  acquisition  and  technology  workforce,  at  about 
150,000  people^  and  includes  people  from  science  and  technology 
organizations.  Table  1.1  lists  the  different  acquisition  and  technology 
workforce  occupations  and  the  number  in  each  occupation  based  on  this 
definition. 


‘  P.  L.  101-510,  Nov.  5, 1990. 

^  This  definition  was  based  on  the  study,  “Identification  of  the  Department  of  Defense  Key  Acquisition 
and  Technology  Workforce,”  April  1999,  Jefferson  Solutions,  Washington,  D.  C. 
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Table  1.1:  The  Defense  Acquisition  and  Technoiogy  Workforce  as  of  April  1999 

Acquisition  and  technology  workforce  occupations 

Total  persons 

Engineers  (electronics,  aerospace,  mechanical,  civil,  and  general)  44, 11 7 

Contracting 

19,387 

Management 

15,509 

Business  and  industry 

12,989 

Communications  and  computers 

9,370 

Administration  and  programs 

5,116 

Scientists 

4,476 

Auditing 

3,692 

Financial  management 

3,618 

Procurement  assistants 

2,650 

Mathematics  and  statistics 

2,400 

Purchasing 

2,158 

Supply  program  management 

1,753 

Inventory  management 

944 

Equipment  specialists 

858 

General  supply 

326 

Miscellaneous 

3,698 

Military 

16,378 

Total 

149,439 

Source:  DOD. 

Definitions  aside,  the  DOD  acquisition  workforce  has  been  undergoing 
restructuring  and  downsizing.  The  National  Defense  Authorization  Acts  for 
Fiscal  Years  1996  and  1997  mandated  reductions  in  the  number  of  civilian 
and  military  employees  in  acquisition  organizations.  DOD  estimates  that, 
as  a  result,  the  acquisition  workforce  will  be  25  percent  smaller  at  the  end 
of  fiscal  year  2000  compared  to  fiscal  year  1995.  DOD  understands  that 
effective  training  will  become  even  more  important  as  the  workforce  is 
reduced  and  new  authority  and  responsibility  are  granted  to  those  who 
remain.  We  have  previously  reported  that  decisions  to  restructure  or 
reduce  this  workforce  should  be  linked  to  getting  better  outcomes  from  the 
acquisition  process;  doing  otherwise  would  miss  an  opportunity  to  address 
the  deep-seated  causes  of  acquisition  problems.® 

Defense  Acouisition  Organizations;  Linking  Workforce  Reductions  With  Better  Program  Outcomes 

(GAO/T-NSIAD-97-140,  Apr.  8, 1997). 
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Althou^  weapon  system  program  offices  comprise  a  subset  of  this 
workforce,  they  are  a  highly  leveraged  subset.  In  planning,  managing,  and 
executing  acquisition  programs,  program  management  offices  are 
responsible  for  about  $80  billion  of  DOD’s  annual  research,  development, 
and  procurement  funds — ^about  30  percent  of  the  fiscal  year  1999  DOD 
budget.  They  influence  much  of  the  work  of  the  rest  of  the  acquisition 
workforce  because  they  are  bringing  new  equipment  into  the  inventory, 
which  must  be  managed,  budgeted  for,  maintained,  and  supplied.  In  a 
sense,  they  are  a  point  of  entry  for  DOD  acquisitions.  In  addition,  these 
offices  are  made  up  of  a  cross  section  of  people  that  draw  from  most  of  the 
acquisition  and  technology  workforce  occupations  cited  in  table  1.1.  As 
such,  they  are  a  medium  in  which  the  training  of  new  initiatives  in  different 
career  fields  converge. 

The  primary  responsibility  for  training  the  acquisition  workforce  in 
general — ^and  new  initiatives  in  particular — falls  within  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition  and  Technology.  That  office  is 
responsible  for  setting  the  standards  that  the  workforce  must  meet.  DAU, 
which  is  responsible  for  designing  and  conducting  the  training  to  meet 
those  standards,  also  reports  to  that  office.  These  and  other  organizations 
responsible  for  setting  training  standards  and  providing  training  are  shown 
in  figure  1.1. 
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Figure  1.1:  Organizations  Responsible  for  Training  Development 


The  Under  Secretary  of  Defense  for  Acquisition  and  Technology  delegated 
responsibility  for  developing  career  paths  and  establishing  educational 
standards  to  the  Office  of  the  Director  of  Acquisition  Education,  Training, 
and  Career  Development.  That  office  is  supported  by  four  functional 
boards  that  have  established  experience,  education,  and  training  standards 
for  the  acquisition  workforce  and  for  professional  certification  levels  in 
each  of  the  acquisition  career  fields.  A  DOD  manual,  DOD  5000.52M, 
“Acquisition  Career  Development  Program,”  implements  and  prescribes 
procedures  for  career  development  of  the  acquisition  workforce.  The 
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manual  establishes  experience,  education,  and  training  standards  for  3 
certification  levels  in  each  of  the  11  acquisition  career  fields.  Level  I  is 
basic  or  entry  level,  level  II  is  the  intermediate  or  journeyman  level,  and 
level  III  represents  the  advanced  or  senior  level  The  specific  training  to 
increase  competency  or  attain  higher  certification  levels  within  a  career 
field  is  often  referred  to  as  functional  training.  There  are  also  standards  for 
specific  acquisition  workforce  positions  (such  as  program  managers), 
position  categories,  and  membership  in  the  acquisition  corps. 

DAU  was  created  by  DAWIA  to  develop  the  training  curriculum  to  meet 
these  standards  and  to  coordinate  the  efforts  of  its  consortium  of  13 
DOD-wide  and  service  schools  that  conduct  the  training  courses.  The 
Defense  Systems  Management  College,  a  DOD-wide  school,  is  dedicated  to 
providing  acquisition-related  training.  Other  member  schools,  including 
those  run  by  individual  military  services,  provide  a  variety  of  training,  of 
which  acquisition  training  is  only  a  part.  While  the  service  schools  provide 
information  on  new  initiatives,  they  do  not  have  primary  responsibility  for 
educating  the  acquisition  workforce.  Each  service  also  has  an  acquisition 
reform  office  that  helps  make  people  aware  of  the  latest  practices  and 
initiatives  that  apply  to  acquisitions,  although  this  office  does  not  play  a 
significant  role  in  designing  or  providing  the  training.  The  Acquisition 
Reform  Communications  Center  is  an  organization  related  to  DAU  that  has 
the  mission  of  sharing  knowledge  about  acquisition  reform  by  providing 
and  disseminating  information  on  how  DOD  is  changing  the  way  it  acquires 
goods  and  services. 


Objectives,  Scope, 
Methodology 


The  Chairman  and  the  Ranking  Minority  Member,  Senate  Committee  on 
Armed  Services,  Subcommittee  on  Readiness  and  Management  Support, 
asked  us  to  review  how  DOD  is  training  its  acquisition  workforce  to 
implement  best  practices  for  acquiring  weapon  systems.  The 
Subcommittee’s  request  is  part  of  a  broader  interest  in  seeing  that  best 
practices  are  incorporated  into  DOD’s  acquisition  process  as  a  way  of 
saving  money  for  modernization,  increasing  efficiency,  and  improving 
quality.  The  objectives  of  this  report  are  to  assess  (1)  the  contribution  DOD 
training  makes  to  program  offices  that  are  applying  best  practices,  (2)  the 
different  training  methods  DOD  and  leading  commercial  firms  use  in 
providing  training  on  practices,  and  (3)  the  different  strategic  approaches 
that  underlie  the  training  methods  DOD  and  leading  commercial  firms  use 
in  implementing  practices. 
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To  meet  these  objectives,  we  focused  on  five  specific  practices  and 
identified  program  offices  recognized  as  being  leaders  in  applying  them. 
We  chose  the  five  practices  based  on  discussions  with  DOD  and  on  our 
previous  work  in  the  application  of  best  practices  to  weapon  acquisitions. 
The  first  four  are  formal  DOD  initiatives  that  are  based  on  best  commercial 
practices,  while  supplier  relationships  is  a  best  practice  we  observed  in 
leading  firms  and  in  some  leading  weapon  system  programs.^  The  selected 
practices  are  the  following: 

•  Post  fl.s  an  independent  variable  C CAIW):  An  acquisition  management 
practice  in  which  aggressive  life-cycle  cost  goals  are  achieved  through 
trade-offs  between  requirements  and  performance. 

•  Integrated  product  teams  YIPT^:  Teams  composed  of  members  from 
functional  disciplines  such  as  engineering,  test,  and  contract 
management.  All  members  contribute  their  particular  expertise  to  team 
decisions  and  to  resolve  issues. 

•  Performan  ce  snecifications:  States  requirements  in  terms  of  required 
results  without  stating  the  methods  to  achieve  those  results.  They 
define  the  functional  requirement  for  operation,  interface,  and 
interchange  characteristics,  and  have  criteria  for  verifying  performance 
compliance. 

•  Past  nerformance:  Information  on  a  contractor’s  past  performance  on 
relevant  prior  work  is  used  as  a  factor  in  source  selection.  The 
information  is  used  to  evaluate  risk  and  the  potential  for  future 
contractor  success. 

•  Supplier  relationships:  A  commercial  practice  in  which  maximum 
participation  of  Suppliers  and  their  suppliers  is  encouraged  to  promote 
product  excellence.  The  best  suppliers  are  selected  and  supported  in  a 
number  of  ways  that  ensure  a  mutually  beneficial  partnership. 

We  focused  on  weapon  system  program  offices  because  of  the  significant 
role  they  play  in  implementing  best  practices.  Initially,  we  considered 
gathering  information  from  a  cross  section  of  program  offices  that  had  a 
range  of  experience  in  implementing  best  practices.  However,  we  found 
that  a  program  office  that  had  little  or  no  experience  with  a  new  practice 
was  not  in  a  good  position  to  comment  on  the  training  needed  to  implement 
the  practice.  Consequently,  we  selected  program  offices  that  DOD 
considered  to  be  leaders  in  one  or  more  of  the  five  practices. 


<  Hpp  Rpst.  Prat^tirRs;  DOD  Can  Help  SuppUers  Contribute  More  to  Weapon  System  Programg 
(GAO/NSIAD-98-87,  Mar.  1998). 
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We  based  our  selections  on  extensive  consultation  with  DOD  officials  to 
ensure  that  the  program  offices  had  experience  in  implementing  the 
practices  and  thus  were  in  a  good  position  to  comment  on  the  training 
resources  that  enabled  them  to  implement  the  practices.  (See  app.  n  for  a 
description  of  the  programs.)  These  program  offices  represented  best  case 
examples  in  DOD  for  specific  practices,  although  pone  of  the  programs  was 
considered  a  leader  in  all  five  practices.  The  program  offices  selected  were 
the  Advanced  Amphibious  Assault  Vehicle  (AAAV),  Advanced 
Medium-Range  Air-to-Air  Missile  (AMRAAM),  Battlefield  Combat 
Identification  System  (BCIS),  Joint  Air-to-Surface  Standoff  Missile 
(JASSM),  Joint  Strike  Fighter  (JSF)  and  Joint  Surveillance  Target  Attack 
Radar  System-Joint  Tactical  Terminal  (JSTARS-JTT).  Table  1.2  shows 
which  of  the  program  offices  were  considered  leaders  for  the  different 
practices. 


Table  1.2:  Best  Practices  Evaluated  at  Program  Offices 


Performance  Past 

Supplier 

CAIV 

IPT  specifications  performance 

relationships 

AAAV 

X 

X 

X 

AMRAAM 

X 

X 

BCIS 

X 

X 

JASSM 

X 

X 

X 

JSF 

X 

X 

JSTARS-JTT 

X 

X 

Source:  GAO  analysis  of  DOD  information. 


At  the  program  offices,  we  used  structured  questions  to  interview  the  key 
people  responsible  for  implementing  an  individual  practice.  Through  the 
interviews,  we  determined  the  various  sources  they  used  to  develop  the 
knowledge  needed  to  implement  best  practices  and  the  extent  to  which  the 
training  DOD  provided  to  the  program  office  contributed  to  this 
knowledge. 

To  assess  strategy  and  the  methods  DOD  uses  to  train  program  office  staff 
on  the  use  of  best  practices,  we  concentrated  on  the  DAU  training 
organizations  and  resources  established  by  DAWIA  to  provide  for  the 
professional  educational  development  and  training  of  the  DOD-wide 
acquisition  workforce.  We  also  assessed  how  these  resources  met  program 
office  needs  in  implementing  best  practices.  In  addition,  we  assessed  other 
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acquisition  training  sources,  including  service  and  agency  specific  schools 
and  the  services’  acquisition  reform  offices.  However,  our  report  focuses 
on  the  training  approach  for  best  practices  and  is  not  intended  to  describe 
all  DOD  training  practices.  We  met  with  the  Director  of  Acquisition, 
Education,  Training,  and  Career  Development;  the  President  of  the  DAU; 
officials  of  the  Defense  Systems  Management  College;  and  representatives 
from  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology.  We  also  met  with  and  collected  data  from  officials  at  the  Air 
Force,  the  Army,  and  the  Navy.  We  reviewed  the  organizational  structure 
and  responsibilities  for  DAU  consortium  members  and  service 
organizations,  the  process  for  establishing  the  DAU  training  curriculum, 
and  survey  data  on  DOD  training  recipients. 

We  evaluated  the  strategies  and  methods  of  commercial  firms  recognized 
for  their  training  excellence  by  examining  how  these  leading  companies 
used  training  to  implement  new  practices  at  their  program  management 
organizations.  To  identify  these  companies,  we  conducted  literature 
searches,  consulted  with  and  collected  data  from  professional  associations, 
and  spoke  with  university  faculty  specializing  in  corporate  organizations. 
We  visited  the  following  four  companies  recognized  as  being  leaders  in  the 
area  of  training; 

•  The  Boeing  Company  is  the  largest  manufacturer  of  commercial 
jetliners  and  military  aircraft  with  234,000  employees  worldwide. 
Boeing’s  Employee  TVaining  and  Development  organization  is 
responsible  for  training  all  Commercial  Airplane  Group  employees, 
approximately  97,000.  There  are  500  to  600  employees  in  the  office,  with 
roughly  300  serving  as  trainers. 

•  Ford  Motor  Company  is  one  of  the  largest  U.S.  manufacturers  of 
automobiles,  trucks  and  provider  of  automotive  services  with  345,000 
employees  worldwide.  Ford’s  Product  Development  Process 
Leadership  organization  was  created  to  provide  assistance,  including 
training,  to  engineers  implementing  the  Ford  Production  Development 
System  (FPDS). 

•  International  Business  Machines  (IBM)  Corporation  is  one  of  the 
world’s  top  providers  of  computer  hardware  and  software  with  290,000 
employees  worldwide.  Different  internal  organizations  provide  training 
that  serves  employees  worldwide,  including  Learning  Services  and  the 
Center  for  Excellence. 

•  Motorola  is  one  of  the  world’s  leading  providers  of  wireless 
communications,  semiconductors,  advanced  electronic  systems,  and 
services  with  over  150,000  employees  worldwide.  Motorola  University 
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has  a  staff  of  over  600  operating  through  account  managers  that  are 
assigned  to  each  business  unit  to  provide  a  “one-face”  education  and 
training  contact. 

These  companies,  recognized  as  industry  leaders,  place  strong  emphasis  on 
training.  At  these  companies,  we  reviewed  company  documents  and 
training  data  and  met  with  individuals  responsible  for  designing  and 
developing  programs  to  educate  and  train  employees  on  major  new 
practices.  We  also  met  with  representatives  from  major  program  offices 
who  were  involved  in  key  training  decisions  and  were  recipients  of  the 
training.  Our  report  highlights  the  best  commercial  training  approaches  for 
implementing  key  new  practices.  As  such,  they  are  not  intended  to 
describe  all  commercial  training  practices  or  suggest  that  commercial  firms 
are  without  flaws. 

Finally,  we  reviewed  several  studies  on  DOD’s  training  organizations  and 
methods.  We  used  these,  as  well  as  the  previous  information  and  analysis, 
to  determine  the  extent  to  which  DOD’s  proposals  to  reshape  DAU  and 
continuous  learning  policy  held  potential  for  better  delivery  of  training  to 
foster  implementation  of  best  practices  by  program  offices. 

We  conducted  our  review  between  April  1998  and  June  1999  in  accordance 
with  generally  accepted  government  auditing  standards. 
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Key  officials  from  weapon  system  programs  at  the  forefront  of 
implementing  best  practices  did  not  find  that  standard  DOD  training 
offerings  provided  the  information  they  needed  to  apply  the  practices  to 
their  programs.  In  evaluating  their  key  sources  of  knowledge  for 
implementing  best  practices,  none  of  the  program  officials  ranked  required 
DOD  training  first,  with  many  ranking  it  last.  DOD  training  either  did  not 
reach  the  right  people  when  it  was  needed  or  did  not  reach  them  at  aU. 
When  training  on  best  practices  was  received,  it  did  not  contain  the  depth 
or  practical  insights  program  office  staff  needed  to  implement  the 
practices.  It  was  primarily  through  their  own  efforts— learning  on  the  job, 
finding  external  training,  or  developing  their  own  training  program— that 
they  attained  the  knowledge  needed  to  apply  best  practices. 

Programs  that  became  leaders  in  applying  best  practices  did  so  primarily 
because  their  managers  realized  that  the  practices  were  key  to  the 
programs’  success.  In  so  doing,  the  managers  were  able  to  identify  what 
knowledge  they  needed  to  apply  the  practices.  Thus,  their  success 
depended  on  having  the  foresight  to  see  what  was  needed,  the  ingenuity  to 
find  good  sources  of  knowledge,  including  training,  and  the  resources 
needed  to  attain  that  knowledge.  Replicating  this  approach  broadly  on 
other  programs  is  problematic.  Other  managers  may  not  realize  the 
significance  of  a  practice  to  the  success  of  their  programs  and  the  need  for 
additional  training.  Also,  they  may  be  uncertain  about  testing  new 
initiatives  on  their  programs.  Some  may  recognize  a  practice’s  importance 
but  be  unable  to  fund  their  own  training  efforts  and  be  left  relying  on 
standard  DOD  training. 


DOD  Training  Did  Not 
Reach  the  Right  People 
at  the  Right  Time 


For  training  to  facilitate  the  adoption  of  a  new  practice,  it  must  be  received 
by  those  responsible  when  they  are  ready  to  begin  implementing  the 
practice.  Program  officials  stressed  that  on  a  new  initiative  or  practice, 
training  must  begin  when  a  practice  is  to  be  introduced.  If  training  is 
provided  too  late,  people  will  be  forced  to  devise  their  own  means  of 
getting  the  knowledge  needed  to  begin  implementation  or  risk  improper 
implementation.  If  training  is  provided  too  soon,  knowledge  could  fade 
before  it  is  applied  at  the  workplace.  For  the  programs  we  reviewed,  those 
responsible  for  managing  or  implementing  best  practices  noted  that  in 
some  cases,  training  did  not  exist  when  they  began  implementing  the 
practices;  in  other  cases,  the  training  was  received  too  late  for  the  job  at 
hand.  Some  were  missed  altogether  by  DOD  training  offerings  because 
they  were  too  senior  to  be  required  to  take  courses,  training  was  not 
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provided  locally,  or  they  fell  outside  the  definition — ^and  training 
curriculum — of  the  acquisition  workforce. 


Training  Came  at  the  Wrong  Program  office  officials  believed  it  is  essential  that  training  on  new 
Tune  for  Some  People  initiatives  accompany  implementation  to  explain  what  the  initiatives  are 

and  to  come  up  with  a  common  understanding  and  way  to  tackle 
program-specific  issues.  For  the  program  offices  leading  the  way  in  best 
practices,  officials  reported  relevant  DOD  training  was  not  offered  at  the 
time  implementation  began.  For  example,  the  BCIS  program  applied 
performance-based  specifications  soon  after  the  1994  DOD  initiative  was 
announced.  However,  specific  training  on  performance  specifications  had 
not  been  developed.  The  AAAV  program  had  already  implemented 
performance  specifications  without  DOD  provided  training.  A  top  manager 
for  JSF  similarly  reported  that  there  was  no  training  for  program  office 
staff  when  they  started  to  implement  CAIV.  There  were  no  guiding 
documents  and  no  one,  including  the  training  community,  knew  what  CAIV 
was.  Similarly,  on  the  JSTARSnJTT  program,  CAIV  training  was  not 
available  when  the  program  office  began  applying  the  practice  in  1995. 
Now,  some  training  is  available,  but  it  was  not  when  it  was  needed  for  the 
program.  Nor  was  training  available  when  the  JASSM  program  office 
began  assessing  contractor  past  performance.  Training  could  have  helped 
avoid  a  protest  of  the  prime  contract  award,  according  to  an  official. 
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Implementation  of  CAIV  on  the  JSF  program  began  with  a  clean  slate— no  advance  training. 

Source:  DOD,  artist  rendition. 

Training  can  also  come  too  early.  A  program  official  questioned  the  staffs 
ability  to  retain  the  information  when  training  is  not  provided  at  a  practical 
time  for  the  assignment.  For  example,  he  noted  that  symposiums  are  good 
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ideas,  but  people  may  not  have  an  opportunity  to  apply  the  ideas  at  the  time 
and  may  not  remember  when  they  need  to. 


Key  People  Were  Missed  by  Applying  best  practices  on  a  weapon  system  program  involves  reaching  not 

Training  on  Best  Practices  program  office  staff  but  also  other  members  of  the  acquisition 

°  workforce,  DOD  people  outside  the  acquisition  workforce,  and 

contractors.  Experience  on  the  programs  we  reviewed  shows  that  it  is 
hard  to  reach  all  of  these  people  with  best  practices  information  through 
standard  DOD  training. 

The  promotion  of  many  best  practices  took  place  after  the  implementation 
of  DAWIA  course  requirements  so  that  senior  people  have  not  taken  formal 
acquisition  training  that  includes  exposure  to  best  practices.  Several  of  the 
senior  program  officials  we  spoke  with  did  not  receive  training  on  new 
practices  because  they  had  not  taken  courses  that  incorporated  best 
practices.  These  officials  had  been  acquisition  officials  for  many  years 
before  DAWIA  and  were  grandfathered  into  their  certification  level.  Their 
job  experiences  were  applied  to  meet  professional  certification 
requirements  and  thus  they  did  not  have  to  take  any  DAWIA  training.  For 
example,  an  official  who  helped  establish  JSF  performance  specifications 
noted  that  he  had  his  last  class  3  years  earlier,  a  contracting  refresher 
comrse.  The  course  may  have  had  a  short  section  on  performance 
specifications,  but  he  could  not  remember  it. 


Training  misses  some  program  office  people  because  it  is  not  offered 
locally.  According  to  program  officials,  training  needs  to  be  conducted  at 
the  local  level  by  subject  matter  when  it  is  needed.  This  is  especially  true 
for  small  programs,  for  which  it  is  difficult  to  spare  people  from  the 
programs  for  long  periods.  One  official  from  a  small  program  said  that 
although  senior  management  stresses  the  importance  of  training,  it  is 
difficult  to  attend  classes  that  run  over  14  weeks.  He  added  that  these 
classes  have  not  been  offered  locally.  Another  program  official  asked  why 
DOD  training  organizations  do  not  bring  some  of  the  harder  to  get  courses 
to  the  field,  as  is  done  by  private  organizations. 
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JSTARS-JTT  officials  believe  training  could  be  improved  by  being  provided  on-site. 
Source:  DOD. 


Training  is  not  reaching  people  outside  program  offices  that  also  play  key 
roles  in  successfully  implementing  best  practices.  For  example,  program 
offices  are  typically  supplemented  with  people  from  separate  functional 
organizations,  such  as  engineering  directorates.  These  people  may  not 
receive  training  on  new  initiatives  in  their  home  organizations  and  can  be 
unfamiliar  with  the  initiatives  when  they  come  to  a  program.  People  that 
set  requirements  early  in  the  process  and  those  in  logistics  must  also  be 
knowledgeable  and  committed  for  practices  to  be  successful.  Exercising 
flexibility  in  requirements,  for  instance,  is  critical  to  the  success  of  CAIV 
and  performance  specifications.  The  need  for  training  was  extended 
further  to  those  that  play  a  role  in  a  weapon  system’s  approval.  Officials 
from  one  program  reported  that  they  had  worked  with  a  prime  contractor 
to  streamline  contract  reporting  requirements  down  to  four  items,  in  line 
with  acquisition  reform.  However,  based  on  a  review  by  a  separate  office 
with  approval  authority  over  the  contract,  40  contract  requirements  were 
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added,  returning  the  contract  to  traditional  reporting  methods.  The 
program  official  said  that  the  reviewing  official  did  not  know  what  the 
program  office  was  trying  to  accomplish  and  did  not  care. 

Prime  contractors  and  subcontractors  are  also  essential  to  the  application 
of  best  practices  but  are  not  part  of  the  DOD  defined  acquisition  workforce 
or  otherwise  targeted  for  training  on  best  practices.  Several  program 
officials  we  met  with  believed  that  some  means  of  informing  contractors  is 
needed  to  make  them  aware  of  important  DOD  initiatives.  Contractors  do 
not  necessarily  understand  or  know  how  to  implement  new  DOD 
initiatives;  for  example,  they  may  not  know  how  an  integrating  IPT  or 
overarching  IPT  works.  While  DOD  is  not  responsible  for  training 
contractors,  it  does  have  to  ensure  that  contractors  understand  best 
practices,  as  well  as  give  them  an  opportunity  to  help  shape  how  these 
practices  are  appUed  on  programs.  The  AAAV  program  management 
ensured  their  prime  contractor  was  knowledgeable  of  best  practices  by 
requiring  training  for  contractor  and  program  office  staff  in  the  prime 
contract. 


Program  officials  reported  that  standard  DOD  training  did  not  prepare 
them  well  for  implementing  the  practices  at  the  workplace.  DOD’s  training 
typically  provided  only  an  awareness  of  the  practices,  not  the  knowledge 
that  is  needed  for  actual  implementation.  Also,  the  training  was  not 
tailored  to  allow  program  offices  to  see  how  individual  practices  applied  to 
their  specific  programs.  Further,  program  officials  noted  that  trainers  did 
not  have  the  practical  experience  to  share  and  were  not  up  to  date  with  the 
most  recent  examples  of  programs  that  had  implementation  experience. 

Program  officials  stated  that  they  need  to  go  beyond  the  theoretical 
concepts  covered  in  most  training  courses.  They  believed  that  the  “how  to” 
is  missing  on  all  the  initiatives  and  that  they  need  to  know  how  to  move 
from  traditional  practices  to  the  new  practices.  DOD  courses  were 
described  in  general  as  too  esoteric  and  not  relevant  to  the  tasks  at  hand. 
For  example,  a  JASSM  official  noted  that  a  contractor’s  past  performance  is 
often  equated  with  DOD’s  Contractor  Performance  Assessment  Rating 
System,  but  it  involves  much  more.  The  JASSM  program  office  evaluated 
contractors’  past  products,  and  in  doing  so,  learned  about  the  quality  of 
their  design,  management,  and  production  processes.  Program  officials 
said  they  went  well  beyond  the  performance  assessment  rating  system 
covered  in  standard  training  courses.  Nonetheless,  they  said  that  they 
could  not  avoid  a  protest  by  the  losing  contractor.  They  believed  they 
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needed  guidance  on  how  to  collect  past  performance  information,  which 
was  not  covered  by  training. 

Program  officials  believed,  overall,  that  training  should  be  designed  more 
for  the  customer  because  acquisitions  are  unique  and  that  prograitrs  may 
have  different  implementation  issues  based  on  program  size,  stage  of  the 
acquisition,  or  type  of  weapon.  A  senior  AAAV  official  said  general  courses 
can  provide  information  about  individual  practices,  but  to  implement  a 
practice,  the  training  must  be  tailored  to  the  program.  JSF  and  BCIS 
officials  concurred.  Some  program  officials  have  observed  that  training 
courses  overemphasize  the  application  of  new  practices  for  larger  and 
newer  programs  and  believe  that  not  enough  coverage  is  given  to  teaching 
how  the  new  practices  apply  to  programs  that  are  smaller  or  older.  For 
example,  officials  noted  that  newer  programs  have  an  advantage  in  starting 
with  a  clean  slate  and  that  training  should  also  show  how  the  practices 
apply  to  older  programs  and  the  benefits  to  be  obtained. 


AMRAAM  officials  believe  training  should  help  program  managers  in  deciding  how  initiatives  apply  to 
their  particular  circumstances. 

Source:  DOD. 
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Program  officials  also  found  the  presentation  methods  for  training  on  best 
practices  were  not  helpful  in  applying  the  practices  in  the  workplace.  They 
believed  that  instructors  lacked  practical  experience  and  current 
information.  For  example,  a  JSF  manager  reported  that  the  executive 
course  he  took  in  1997  on  CAIV  was  not  valuable  because  the  instructors 
had  no  experience  with  CAIV  and  did  not  know  how  to  explain  it.  Another 
program  official  thought  that  instructors  were  left  in  place  too  long  and  had 
only  old  experiences  to  share.  Officials  noted  that  course  material  was 
frequently  out  of  date  and  sometimes  incorrect.  For  example,  one  JASSM 
program  official  attended  an  engineering  course  in  which  the  instructor’s 
answer  on  CATV  was  wrong.  Another  said  the  acquisition  reform  example 
the  instructor  used — ^from  the  official’s  own  program — ^was  about  2  years 
behind  what  was  being  applied  on  the  program.  The  official  brought  up 
more  current  examples  of  practices  being  used  on  the  program,  but  the 
instructor  did  not  want  to  discuss  them. 

Training  also  suffered  from  limited  use  of  case  studies  that  would  allow 
students  to  see  how  an  initiative  in  the  abstract  might  apply  to  their  own 
programs.  Officials  from  several  programs  added  that  current  case 
histories  should  show  the  application  of  acquisition  principles  in  a  program 
context.  Case  studies  would  include  the  successes,  as  well  as  the  pitfalls 
and  solutions.  'The  case  studies  should  be  designed  for  the  customer, 
another  said,  and  be  applied  at  multiple  levels.  For  example,  programs 
could  benefit  if  training  allowed  students  to  play  off  the  risks  and  benefits 
of  how  new  initiatives  apply  to  their  particular  program.  Real  case 
experiences  help  others  visualize  how  practices  could  apply.  BCIS  officials 
reported  that  during  the  presentation  of  their  case  study  for  the  1998 
Acquisition  Reform  Week,  they  realized,  as  they  explained  what  they  did  to 
implement  initiatives,  that  the  audience  gained  better  insight  on  the  issues. 
Another  official  said  he  shared  his  program’s  experiences  with  those  in  his 
class,  but  those  in  other  classes  would  not  have  the  benefit  of  lus 
experiences. 

Program  officials  did  not  believe  that  there  was  an  effective  means  for 
providing  feedback  on  the  quality  and  usefulness  of  courses,  such  as  the 
need  to  update  course  materials,  or  course  relevance.  'They  would  like  to 
see  the  students — ^the  practitioners — have  more  impact  on  what 
a(^justments  are  needed  to  courses.  'The  only  means  of  feedback  program 
office  people  were  aware  of  was  the  end  of  class  survey  form.  However, 
these  surveys  did  not  allow  students  to  give  in-depth  feedback  or  ask  the 
questions  that  got  to  the  larger  issue  of  whether  students’  training  needs 
were  met. 
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Limitations  of 
Standard  Training  Led 
Program  Offices  to 
Develop  Their  Own 
Training  Solutions 


In  evaluating  their  key  sources  of  knowledge  for  implementing  best 
practices,  none  of  the  program  officials  ranked  required  DOD  training  first, 
with  many  ranking  it  last.  To  obtain  the  knowledge  needed  to  implement 
key  practices,  officials  in  leading  programs  developed  their  own  solutions. 
They  had  a  vision  of  what  they  wanted  to  accomplish  and  they  devised  a 
variety  of  methods,  such  as  funding  training  beyond  standard  DOD 
offerings,  creating  their  own  internal  training  programs,  and  learning  on  the 
job.  Strong  program  managers,  supported  by  the  executives  above  them 
and  strong  working  relationships  with  their  contractors,  were  key  to  the 
implementation  of  the  practices.  Program  managers  of  the  leading 
programs  cautioned,  however,  that  not  all  officials  have  the  vision  or  the 
resources  to  mirror  this  approach. 


Implementation  Success 
Tied  to  Vision  and  Support 
at  All  Levels 


All  the  leading  programs  had  at  least  one  element  in  common — strong 
leadership  committed  to  implementing  practices  that  would  help  their 
programs  succeed.  Leaders  were  described  as  having  vision  and  knowing 
what  they  had  to  do  to  realize  that  vision.  This  included  accepting  the  risks 
for  trying  new  approaches.  For  example,  the  original  AAAV  program 
manager  conceived  of  the  unique  approach  of  collocating  the  program 
office  and  the  contractor  because  he  thought  it  was  essential  to  making 
IPTs  work.  IPTs  became  the  backbone  of  the  AAAV  program. 
Consequently,  team  training  was  viewed  as  so  important  that  it  was 
incorporated  into  the  prime  contract  as  a  requirement.  Support  for 
collocation  and  the  team  approach  came  from  top  management  of  both  the 
contractor  and  the  program  office.  According  to  program  officials,  it  does 
not  make  any  difference  how  good  the  training  is  without  management 
support;  Junior  people  can  come  back  from  training  with  new  ideas  and 
have  them  go  nowhere  if  they  do  not  get  management  support. 
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Training  for  AAAV  was  tallorGd  to  support  contractor  and  program  office  collocation. 


Source:  DOD. 

In  turn,  managers  point  to  the  support  that  they  receive  from  their  senior 
management  as  an  important  factor  in  their  ability  to  be  successful.  One 
official  described  his  manager  as  aggressive  in  his  efforts  to  pave  the  way 
for  trying  new  practices.  To  do  that,  the  senior  manager  said  that  he  is 
willing  to  go  up  against  tire  established  bureaucracy  and  provide  cover  for 
his  program  managers  to  try  new  things.  One  program  manager  cited  a 
personal  commitment  to  a  reform-minded  DOD  official  as  part  of  his 
motivation  for  making  acquisition  reform  work.  He  said  that  with  his 
bosses’  support,  he  can  do  what  he  thinks  is  right  on  his  program,  such  as 
making  past  performance  central  in  selecting  a  prime  contractor.  However, 
he  noted  that  other  managers  have  not  had  that  ability  because  they  are  not 
supported  when  they  propose  doing  things  differently. 

Program  managers  believe  that  a  key  element  for  adopting  best  practices 
was  developing  trust  or  partnership  with  the  contractor.  An  AMRAAM 
manager  believes  the  ability  to  make  radical  program  changes  has 
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depended  on  working  closely  with  the  prime  contractor,  and  seeing  that  the 
same  relationship  exists  between  the  prime  contractor  and  subcontractors. 
The  JASSM  office  organized  its  IPTs  to  mirror  the  structure  of  the 
contractor’s  IFIfe,  rather  than  along  functional  lines  like  most  offices.  A  top 
JASSM  official  credited  the  high  level  of  trust  between  the  program  office 
and  the  contractor,  along  with  the  program  office’s  creativity  and 
innovation,  for  enabling  their  use  of  new  practices.  This  perspective  is 
essential  to  managing  programs  in  today’s  environment  and  to  make  teams 
work,  another  official  noted,  but  it  is  not  taught  in  standard  DOD  training 
offerings. 

More  managers  would  attempt  to  apply  best  practices  if  training 
encouraged  it,  a  program  official  said,  particularly  the  training  provided  for 
junior  program  managers.  More  general  training  is  needed  to  support  a 
new  program  environment  for  adopting  new  practices  that  goes  beyond  the 
current  training  emphasis  on  the  mechanics  of  management  and  theory. 
Program  officials  reported  that  new  program  manager  training  as  recent  as 
1997  did  not  reflect  the  new  approach  they  have  practiced  on  their 
programs.  One  believed  that  it  was  a  lost  opportunity  for  DOD  to  not 
impress  upon  new  managers  their  role  in  adopting  new  practices  and  the 
potential  benefits. 


Leading  Program  Officials 
Innovated  to  Get  Needed 
Training 


With  the  support  of  senior  management,  program  officials  used  a  mixture 
of  strategies — generally  outside  of  standard  DOD  training  offerings — to  get 
the  knowledge  they  needed  to  apply  best  practices.  A  number  of  programs 
used  their  own  resources,  including  cumulative  job  expertise  of  the  staff 
and  personal  research  to  learn  how  to  implement  new  initiatives.  For 
example,  JSF  officials  stated  they  learned  how  to  apply  CAIV  by  organizing 
warfighters,  engineers,  and  analysts  together  in  a  team.  They  learned  as 
they  went,  developing  materials  on  their  own.  The  AMRAAM  program  had 
a  young  staff  member  gather  information  on  CAIV  from  various  sources 
such  as  the  internet  and  conferences.  From  that  starting  point,  they 
learned  as  they  went  along.  In  setting  performance  specifications,  it  was 
the  personal  experience  of  AMRAAM  officials,  coupled  with  commitment, 
that  allowed  the  specifications  to  be  set  at  a  performance,  rather  than 
detailed,  level.  Despite  their  own  experiences,  program  officials  did  not 
recommend  the  leam-as-you-go  approach;  everyone  should  not  have  to 
reinvent  the  wheel. 


Programs  officials  used  their  own  program  funds  to  go  beyond  standard 
DOD  training  by  sending  staff  to  nongovernment  training  or  to  bring 
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experts  in.  For  example,  frustrated  with  DOD  training,  the  JASSM  program 
manager  sends  people  to  outside  training,  such  as  Harvard  leadership 
courses,  to  have  them  learn  and  grow  beyond  the  basic  training  and 
develop  creative  and  innovative  thinking.  He  has  his  staff  take  80  hours  of 
general,  nonfunctional  training.  Another  manager  said  some  of  his  best 
training  was  from  off-site  sessions  sponsored  by  JASSM  that  dealt  with 
people  issues  that  were  critical  to  making  IPTs  work.  JSF  officials  took  a 
private  sector  course  on  performance-based  specifications  and  used  a 
model  developed  by  their  contractor.  BCIS  officials  hired  an  outside  firm 
to  teach  program  officials  on  two  occasions,  both  of  which  included  a  CAIV 
component. 


Figure  2.5:  JASSM 


JASSM  officials  sponsor  training  to  develop  staff  leadership  skills. 
Source;  DOD. 


AAAV  officials  developed  their  own  training  program  tailored  to  the 
program’s  work  environment  and  training  needs.  A  key  characteristic  of 
the  program  is  the  collocation  of  government  and  contractor  program 
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officials,  making  team  dynamics  an  important  factor.  The  prime  contract 
specified  that  teams  must  be  used,  and  IPT  training  was  charged  to  the 
contract.  The  contractor  hired  a  third  party  to  develop  a  training  program 
tailored  to  the  AAAV  program,  and  program  office,  contractor,  and 
subcontractor  staff  were  taught  together,  on-site  as  a  team.  Joint  training 
was  used  to  establish  a  common  culture  for  participants.  TVaining  was 
given  one  time  to  the  team  initially  and  then  to  every  new  person.  The  team 
training  started  with  mandatory  10  hours  of  IPT  training,  with  sessions 
covering  the  AAAV  program,  trust,  communications,  and  other  team 
dynamics.  CAIV  training  was  added  as  part  of  the  training  for 
implementation  within  the  team  context. 

The  team  approach  to  training  was  supported  by  other  leading  programs  as 
well.  On  the  JASSM  program,  the  manager  reported  that  he  just  had  his 
entire  office  (about  30  staff)  take  1  week  of  leadership  training,  which 
included  topics  such  as  stress  management,  critical  thinking,  and 
decision-making.  JSF  officials  took  the  “train  the  trainer”  approach  and 
developed  a  team  of  experts  within  JSF  and  then  the  experts  trained 
everyone  that  would  be  involved  in  implementing  performance 
specifications.  The  experts  used  multiple  sources  to  train  themselves,  such 
as  published  guidance  and  talking  with  their  peers.  They  then  developed 
basic  guidance  for  the  staff,  such  as  engineers,  who  needed  a  common 
understanding  on  how  to  write  requirements  at  a  performance  level. 


Other  Programs  May  Lack 
Leading  Programs’  Ability  to 
Innovate 


Program  officials  reported  that  they  were  fortunate  to  have  staff  that  could 
use  their  collective  experiences  to  work  through  problems  in  implementing 
initiatives.  However,  they  noted  that  not  all  programs  will  have  the  same 
advantages.  JASSM  officials  said  top  service  officials  handpicked  program 
officials  for  the  program  team  because  their  personal  characteristics 
supported  flexibility  and  creativity.  Similarly,  JSF  officials  reported  that 
the  program  office  was  staffed  with  multifaceted  people,  as  weU  as  strong, 
senior  management  support  and  upfront  money  for  training.  They  noted 
that  smaller  programs  may  be  staffed  by  junior  officials  with  less 
experience  to  draw  upon  or  be  unable  to  devote  staff  to  research 
information  on  how  practices  might  apply  to  their  situation.  Limited 
funding  may  also  be  an  issue  for  some  programs.  As  one  program  official 
said,  his  program  office  could  pay  for  external  training,  but  for  many 
programs,  the  cost  would  be  an  impediment  since  training  is  one  of  the  first 
items  cut  in  a  program  budget.  Consequently,  smaller  programs  might  need 
to  rely  more  heavily  on  standard  DOD  training  as  their  main  source  of 
information. 
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Leading  commercial  firms  and  DOD  apply  different  training  methods  to 
implement  key  practices.  Commercial  firms  use  targeted,  hands-on 
methods  that  include  conducting  a  front-end  analysis  to  determine  the 
teams’  training  requirements  and  regularly  involve  those  implementing  the 
practices  in  making  important  training  decisions  to  ensure  program  teams 
are  trained  on  key  practices.  Thus,  the  training  is  customized  to  meet  the 
specific  needs  of  the  teams.  These  methods  also  involve  providing  many 
hours  of  training — beyond  standard  skill-based  or  functional  training — 
focused  on  the  implementation  of  a  sin^e  practice.  Company  officials 
believe  the  targeted  approach  results  in  more  useful  training — improving 
the  personnel  coverage,  course  depth,  and  timing— which  helps  to  improve 
outcomes  of  the  final  product. 

In  contrast,  DOD  training  on  best  practices  is  delivered  through  traditional 
DAWIA  certification  courses  and  vehicles  such  as  videos  and 
computer-based  training,  which  are  limited  in  reaching  the  right  people  at 
the  right  time  and  in  providing  the  needed  depth  to  implement  best 
practices.  DOD  does  not  have  a  counterpart  to  the  commercial  method  of 
providing  customized,  hands-on  assistance  to  support  program  office  staff 
and  other  implementers  of  key  practices.  Although  exceptions  exist,  there 
is  no  systematic  effort — or  responsible  organizations — ^within  DOD  to 
directly  assist  key  implementers  to  use  new  practices.  Further,  DOD  does 
not  have  a  comprehensive  means  for  allowing  program  staff  and  others  to 
influence  training  decisions  in  a  way  that  could  improve  the  relevance  of 
training. 


Commercial  Firms  Use 
Targeted,  Hands-on 
Methods  to  Improve 
Training  Usefulness 


For  routine  training,  such  as  skill-building,  leading  commercial  firms 
provide  standard  training  offerings,  including  functional  area  courses  and 
instruction  on  corporatewide  issues,  such  as  communications  or  ethics. 
However,  when  implementing  key  new  practices — ^such  as  those  that 
change  product  development  and  production — the  firms  go  beyond  the 
standard  training  offerings.  Commercial  firms  use  a  targeted,  hands-on 
training  approach  to  ensure  program  teams  are  in  a  good  position  to 
implement  a  new  practice.  They  provide  numerous  hours  of  training, 
typically  through  a  single  company  organization,  targeted  to  the 
implementation  of  a  key  practice.  The  practice-specific  training  hours  are 
targeted  to  the  program  teams  most  likely  to  implement  the  new  practice. 
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The  elements  common  to  how  the  four  leading  firms  provide  training  on  a 
key  initiative  or  practice  include 

•  a  front-end  analysis  of  program  teams’  needs  and  training  requirements; 

•  involvement  of  program  teams  in  key  training  decisions; 

•  customized  training  to  meet  program  teams’  specific  needs; 

•  targeted  training  for  the  implementation  of  specific  practices;  and 

•  improved  training  outcomes,  including  better  course  depth,  timeliness, 
and  reach. 

'The  training  organizations  of  leading  commercial  firms  conduct  a  front-end 
analysis  to  determine  the  needs  and  training  requirements  of  program 
offices  implementing  new  practices.  'The  analysis  is  also  used  to  identify 
and  address  barriers  each  program  office  faces  when  implementing  new 
practices.  According  to  the  Director  of  the  Benchmarking  Forum  for  the 
American  Society  for  TVaining  and  Development,  this  type  of  analysis  is 
crucial  for  an  organization  to  be  able  to  institute  performance-improving 
measures.  Using  information  from  the  front-end  analysis,  the  training 
organizations  customize  the  training  to  ensme  that  it  directly  assists 
program  teams  in  implementing  new  practices.  One  company  official  told 
us  that  training  is  costly  and  when  it  misses  the  mark,  the  company  pays  a 
big  price.  Given  the  importance  of  training  when  implementing  a  key 
practice,  company  officials  believe  that  it  is  crucial  to  ensure  that  the 
training  is  beneficial  to  the  key  implementers  of  the  practice.  To  ensure  that 
the  training  will  address  ttie  needs  of  the  program  teams,  the  training 
organizations  involve  the  staff  in  making  important  training  decisions. 
Program  staff  help  decide  the  amount  of  training  to  be  provided  for  certain 
job  positions,  course  objectives,  and  depth  of  course  coverage.  Company 
officials  believe  their  training  approach,  which  includes  program  staff,  has 
resulted  in  the  right  amount  of  course  depth,  timeliness,  and  coverage  of 
personnel  in  the  conunercial  firms.  Following  are  descriptions  of  the 
training  methods  employed  by  companies  on  key  initiatives  or  new 
practices. 

Officials  from  Boeing’s  Employee  Training  and  Development  organization 
state  that  their  primary  goal  is  to  support  their  customers — employees 
assigned  to  the  Commercial  Airplane  Group.  The  training  representatives 
develop  a  partnership  with  the  staff  from  the  beginning  of  the  program  to 
design  and  manufacture  a  new  airplane.  The  training  representatives  form 
“drop  teams”  to  collocate  with  the  program  to  conduct  a  front-end  analysis 
and  learn  as  much  as  possible  about  the  business  process  and  the  staffs 
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concerns.  The  analysis  allows  the  drop  team  to  determine  what  training  is 
needed  to  support  the  staff  implementing  new  practices. 

Boeing  training  officials  said  they  worked  side  by  side  with  the  program 
staff  to  create  a  training  program  that  provided  team  building  and  conflict 
resolution  techniques  and  technical  skills  training  that  specifically  focused 
on  improving  work  competencies  that  would  change  as  a  result  of  the  777’s 
new  digital  environment.  To  ensure  all  777  staff  were  equally  trained, 
employees  were  required  to  complete  training  before  they  reported  to  the 
program.  For  example,  the  professional  employees — engineers  and 
drafters — ^were  required  to  complete  120  hours  of  start-up  training  on 
several  key  777  practices,  including  design  build  teams  and  computer-aided 
three-dimensional  interactive  application^  software.  Teams  were  often 
trained  together  at  the  work  location.  Boeing  officials  stated  that  training 
was  instrumental  to  the  implementation  of  key  practices  on  the  777 
program,  such  as  design  build  teams — essentially  IPTs.  The  officials  stated 
that  design  build  teams  were  at  odds  with  the  company’s  culture  because 
employees  were  not  accustomed  to  working  in  a  team  environment  and 
sharing  information  across  functional  areas. 


*  This  application  is  a  computer-based  design  tool  that  allows  designers  the  opportunity  to  view  design 
drawings  and  the  interface  of  the  millions  of  airplane  parts  as  three  dimensional. 
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Boeing’s  use  of  design  build  teams  created  a  major  culture  change  for  the  staff  assigned  to  the  777 
program. 

Source:  The  Boeing  Company. 


Boeing  officials  believe  their  partnership  approach  improved  their  training. 
Training  representatives  stated  that  the  partnership  resulted  in  program 
management  support,  which  ultimately  led  to  acceptance  from  the  program 
staff.  The  representative  stated  that  since  Boeing  has  involved  program 
staff  in  decisions  regarding  training,  course  “no-show”  rates  have 
decreased.  A  senior  manager  for  program  operations  for  the  777  program 
stated  that  because  he  and  other  senior  program  leaders  drove  key  training 
decisions,  the  training  was  tailored  to  the  staffs  needs  and  provided  the 
necessary  skills  and  orientation  to  work  in  the  new  environment. 


Page  42 


GAO/NSIAD-99-206  Best  Practices 


Ford  Motor  Company 


Chapter  3 

DOD  Training  Methods  for  Best  Practices  Do 
Not  Go  as  Far  as  Leading  Commercial  Firms 


The  Ford  Motor  Company  created  an  organization,  Product  Development 
Process  Leadership,  with  the  singular  purpose  of  supporting  its  100 
program  offices  in  designing  new  or  modifying  vehicle  lines  in  the 
implementation  of  the  FPDS-a  lean  engineering  process.  The  organization 
provides  internal  communication  regarding  FPDS  to  the  teams,  and  its 
training  representatives  work  with  the  teams  to  conduct  an  analysis  to 
learn  first  hand  if  impediments  to  FPDS  implementation  exist.  Other 
support,  such  as  team  coaching,  is  provided  to  facilitate  the  engineering 
team’s  implementation  of  FPDS  in  the  workplace. 


Figure  3.2:  Ford  Focus 


Ford  uses  training  to  Improve  the  timeliness  and  quality  of  new  vehicle  launches. 
Source:  Ford  Motor  Company. 


Ford  officials  told  us  that  their  training  focus  is  to  provide  practical  skills 
just  in  time — that  is,  when  it  coincides  with  the  need  to  apply  the  skills  on 
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IBM 


Motorola  University 


the  job.  Ford  provides  over  50  hours  of  training  to  instruct  engineers  on 
how  to  do  their  jobs  using  the  FPDS  process.  To  that  end,  Ford  uses 
internal  subject  matter  experts — engineers  who  have  been  on  teams 
designing  new  or  modifying  existing  vehicle  lines — ^to  help  tailor  FPDS 
course  topics  to  make  them  relevant  to  the  work  environment.  Ford 
officials  stated  that  the  subject  matter  experts  understand  the  details  of  the 
FPDS  and  are  needed  to  ensure  that  the  training  developed  is  practical  at 
the  working  level.  According  to  the  manager  responsible  for  training 
employees  to  launch  new  vehicles  at  Ford,  his  office  pulls  together  the 
training  that  is  necessary  to  get  the  job  done.  To  further  improve 
practicality  and  timeliness,  the  manager  stated  that  on-the-job-training  at 
the  worksite  is  used  to  the  extent  possible. 

IBM’s  Center  for  Excellence  provides  in-house  education  consultants  to 
personally  understand  the  business  situation  and  product.  The  consultants 
interview  business  unit  staff  to  assess  the  staffs  training  needs  and  identify 
inhibitors  to  implementing  new  practices.  The  assessments  can  take 
2  weeks  to  4  months,  depending  on  the  size  of  the  business  unit.  An  IBM 
trainer  for  Object  Orientation  Project — ^a  companywide  software 
development  practice — explained  that  providing  practical  training  is  one  of 
the  Center’s  guiding  principles.  To  achieve  this  goal,  the  company  first 
stresses  the  importance  of  using  experienced  instructors  who  have 
practical  knowledge  in  “doing”  what  they  are  teaching.  Second,  employees 
are  assigned  to  an  Object  Orientation  Project  before  they  take  the 
5-  to  7-week  immersion  training  comse.  According  to  the  IBM  trainer,  this 
requirement  has  improved  practicality  because  the  students  have  better 
knowledge  retention,  as  opposed  to  having  to  wait  6  months  to  apply  the 
information.  Lastly,  to  further  enhance  practicality  and  relevance,  IBM 
integrates  case  studies  with  real  examples  related  to  the  student’s  next 
assignment  into  the  Object  Orientation  Project  training.  According  to  an 
official  from  IBM’s  Center  of  Excellence,  most  training  is  provided  and 
tailored  to  entire  work  teams  or,  at  a  minimum,  to  individuals  with  common 
responsibilities. 

Motorola  University  focuses  on  providing  training  and  education  solutions 
for  its  business  units.  The  university  recently  began  to  assign  an  account 
management  team  to  consult  and  advise  senior  leadership  for  each 
business  sector  in  order  to  anticipate  and  provide  appropriate  training. 

The  management  team  partners  with  the  business  unit  staff  to  identify  their 
training  needs  for  implementing  new  practices,  such  as  Five  Nines — an 
effort  to  improve  product  reliability  to  the  level  of  99.999  percent.  The 
account  team  works  with  the  business  units  to  assess  their  training  needs 
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and  develop  a  plan  to  meet  those  needs.  This  partnership  enables  the 
university  to  customize  its  training  to  the  specific  needs  of  the  various 
types  of  engineers,  such  as  software  engineers,  within  the  business  units.  A 
software  engineer  stated  that  the  university  provides  separate  training 
courses  tailored  to  meet  the  different  and  often  distinct  needs  within  the 
engineering  community  implementing  Five  Nines.  For  example,  a  software 
engineer  could  receive  up  to  120  hours  of  training  targeted  to  implementing 
Five  Nines.  Although  the  account  management  structure  is  relatively  new, 
the  engineer  observed  that  most  business  unit  staff  are  reacting  favorably 
to  the  effort  because  the  coordinated  approach  provides  one-stop 
shopping. 


DOD  Does  Not  Target 
Training  on  Key 
Practices  to  Program 
Offices 


DOD  does  not  have  a  counterpart  to  the  commercial  hands-on  approach  for 
directly  assisting  key  implementers  of  the  best  practices.  DOD  relies 
primarily  on  its  standard  training,  including  DAWIA  courses,  augmented  by 
videos,  internet-based  training,  satellite  broadcasts,  and  roadshows,  to 
inform  staff  on  best  practices.  These  venues  were  designed  to  focus  on 
functional  training  and  to  increase  the  awareness  of  new  practices.  As 
such,  they  do  not  provide  the  necessary  depth  or  reach  enough  of  the  right 
people  at  the  right  time  to  be  of  help  in  implementing  best  practices  at 
program  offices.  Responsibility  for  training  on  best  practices  is  diffused 
among  several  DOD  organizations,  including  DAU  and  the  service 
acquisition  reform  offices.  We  did  not  find  an  organization  that  was  able  to 
tailor  and  help  deliver  training  on  best  practices  to  the  program  offices  we 
visited.  Furthermore,  DOD  does  not  systematically  involve  program  office 
staff  and  other  implementers  in  key  decisions  regarding  best  practices 
training.  Currently,  no  feedback  mechanism  exists  to  determine  the  effect 
of  DOD’s  training  on  the  implementation  of  new  practices  at  the  program 
office  level. 


DAWIA  Training  Not  Well 
Suited  as  a  Primary  Means 
for  Conveying  Best 
Practices 


DAU  training  is  designed  primarily  for  employees  seeking  career  level 
certification  in  the  acquisition  workforce,  as  required  by  the  DAWIA 
standards.  DAU  incorporates  best  practice  topics  into  the  DAWIA  courses 
as  drop-in  modules  that  provide  a  survey  of  the  topic.  The  information 
conveyed  is  enough  to  provide  a  general  awareness  of  the  concept  but  not 
enough  to  implement  the  practice  at  the  workplace.  While  this  approach 
may  provide  sufficient  information  for  the  target  audience — newer 
acquisition  employees — ^it  has  inherent  limitations  when  it  comes  to 
providing  best  practices’  implementation  training  to  the  entire  acquisition 
workforce.  For  example,  DAU  courses  have  been  developed  by  functional 
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boards  that  teach  skill-based  competencies  for  functional  career  fields 
versus  best  practices.  As  a  result,  DAU  courses  are  primarily  aligned  along 
career  fields,  such  as  engineering  and  cost  estimating.  Because  personnel 
assigned  to  a  particular  functional  area  are  given  priority  for  training  in  that 
function,  program  officials  told  us  that  it  is  difficult  for  personnel  outside 
of  their  areas  to  take  courses.  On  one  program,  an  official  noted  that 
because  of  limited  space,  only  5  to  10  percent  of  the  program  staff  can  take 
training  outside  of  their  functional  areas  each  year. 

One  program  manager  believed  that  the  functional  training,  while 
important  to  career  fields,  was  no  longer  as  relevant  to  the  role  of  today’s 
program  office.  He  noted  that  the  combination  of  best  practices, 
delegation  of  key  responsibilities  to  contractors,  and  fewer  staff  has  altered 
the  program  office’s  role.  In  his  opinion,  the  program  office  used  to  be 
closely  involved  with  managing  the  design  of  a  weapon  system  and 
double-checking  the  prime  contractor’s  work.  In  this  role,  the  program 
office  was  organized  and  operated  along  functional  lines,  and  functional 
training  was  relevant  to  how  a  program  office  operated.  He  observed  that 
today  the  program  office  is  not  as  involved  with  the  designing  of  the 
weapon  system,  nor  is  it  able  to  mirror  the  contractor’s  functions.  Rather, 
the  program  office  must  be  expert  at  what  the  government  can  control — 
which  the  program  manager  referred  to  as  key  leverage  points.  These 
leverage  points  include  the  requirement  trade-off  process,  the  selection  of  a 
prime  contractor,  and  the  establishment  of  key  relationships  that  enable 
the  program  office  staff  to  have  insight  into  the  contractor’s  actual 
progress.  Program  office  staff,  working  in  an  IPT  environment,  must  have 
knowledge  of  multiple  career  fields  and  work  in  a  cross-functional  setting. 
As  a  result,  he  believes  that  functional  training  alone  no  longer  covers  the 
things  most  critical  to  a  program  manager.  The  kind  of  training  that  is 
needed— such  as  on  the  leverage  points— must  be  obtained  elsewhere  by 
the  program  office. 

The  usefulness  of  the  DAU  courses  is  further  hampered  by  limited 
availability,  which  restricts  program  offices  from  receiving  training  when 
needed.  According  to  a  DAU  school  representative,  the  consortia  of 
schools  can  train  about  10  percent  of  the  workforce  each  year. 
Furthermore,  more  senior  staff  may  have  limited  exposure  to  best 
practices  because  the  majority  of  them  have  already  met  training 
requirements  and  are  not  taking  the  certification  courses  that  introduce 
best  practices.  While  these  staff  are  not  prohibited  from  taking 
certification  courses  as  part  of  their  continuing  education  requirements, 
availability  is  limited  and  priority  is  given  to  individuals  seeking 
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certification.  These  inherent  limitations  are  consistent  with  the  training 
shortfalls  noted  by  program  office  staff  in  chapter  2. 


Other  Training  Methods  Are 
More  Dedicated  to  Best 
Practices  but  Are 
Awareness-oriented 


DAU’s  Acquisition  Reform  Communication  Center  (ARCC)  is  a  key  avenue 
for  disseminating  information  on  the  best  practices  to  the  acquisition 
workforce.  ARCC  provides  training  through  videos  and  periodic  satellite 
broadcasts  on  a  variety  of  best  practice  topics.  However,  ARCC  does  not 
track  attendees  and  has  no  assurance  that  the  workforce  adequately 
receives  the  training.  For  those  that  do  attend,  the  introductory  nature  of 
the  training  may  not  provide  the  depth  or  specificity  to  implement  the 
practice  at  the  workplace  or  in  a  time  frame  that  is  helpful. 


The  acquisition  reform  offices  in  the  services  conununicate  best  practice 
information  through  acquisition  reform  courses,  periodic  satellite 
broadcasts,  and  informational  videos,  which  are  sufficient  for  broad 
exposure  on  best  practices  but  again  are  of  limited  depth  for  practical 
application.  Roadshows,  traveling  multiday  training  workshops  provided  to 
staff  at  a  number  of  locations,  and  Acquisition  Reform  Week,  designated  by 
DOD  as  an  opportunity  for  all  service  organizations  to  cease  their  normal 
operations  and  focus  on  acquisition  and  logistics  reform  initiatives,  are  also 
used  to  provide  best  practices  training  to  a  wide  range  of  the  acquisition 
workforce.  These  methods  also  have  limitations  in  depth  and  workforce 
coverage.  For  example,  roadshows  typically  provide  awareness  training  on 
the  practices  and  do  not  provide  in-depth  information  needed  for 
implementation  at  the  workplace.  A  program  official  beheves  that  only 
10  to  15  percent  of  the  acquisition  staff  attend  the  second  day  of  roadshow 
workshops,  where  more  detailed  training  is  provided.  The  annual 
Acquisition  Reform  Weeks  also  provide  awareness  level  training.  Neither 
method  is  tailored  to  specific  program  offices  or  provides  assurance  that  it 
is  delivered  at  the  time  most  needed  by  the  workforce. 


DOD  Training  Organizations 
Not  Set  Up  to  Help  Design 
or  Deliver  Tailored  Training 
to  Program  Offices 


DOD  does  not  have  organizations  that  are  comparable  to  those  in 
conunercial  firms  and  that  work  with  program  offices  in  identifying  best 
practices  applicable  to  a  particular  program,  designing  an  approach  to 
training  the  program  office  staff,  or  deUvering  the  training  to  the  program 
offices.  DOD  training  organizations — those  that  deliver  DAWIA  and 
acquisition  reform  training — are  not  set  up  or  have  the  resources  to  deliver 
best  practices  quickly  and  easily  to  program  staff,  and  then  ensure 
implementation  at  the  working  level. 
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The  DAU  consortium  integrates  best  practices  topics  into  DAWIA 
certification  or  acquisition  reform  training  but  does  not  develop  training 
specifically  for  implementing  the  practices.  Although  DAU  has  a 
substantial  full-time  faculty,  the  faculty  members’  main  priority  has  been  to 
teach  in  the  classroom.  One  course  director  informed  us  that  faculty 
members  are  evaluated  on  the  basis  of  hours  of  training  provided  in  the 
classroom,  which  implicitly  discourages  work  outside  the  classroom,  such 
as  consulting  with  program  offices.  Also,  httle  or  no  consultation  occurs 
between  the  course  designers  and  the  implementers  on  how  to  best 
implement  a  practice  at  the  program  office  level.  The  Director  of  DOD’s 
International  and  Commercial  Systems  Acquisition  unit  stated  that  DOD’s 
current  structure  for  defining  and  developing  training  courses  does  not 
have  clear  accountability  to  ensure  that  training  on  best  practices  is 
provided  at  the  program  level.  Several  program  office  staff  informed  us 
that  they  have  not  been  given  the  opportunity  to  provide  input  to  influence 
the  content  of  the  courses.  Instructors  for  the  courses,  who  do  have 
frequent  contact  with  program  office  staff,  also  believed  that  their  ability  to 
influence  the  content  of  the  courses  was  limited.  DOD  does  use  internet 
site  surveys  to  obtain  staff  feedback  on  acquisition  reform  training,  which 
could  help  tailor  the  courses  more  to  the  needs  of  the  program  offices. 
However,  program  office  staff  believed  the  survey  forms  were  inadequate 
for  addressing  the  specific  training  needs  of  individual  program  offices. 

Organizations  with  more  direct  responsibility  for  providing  training  on  best 
practices  are  staffed  to  the  level  needed  to  design  distance  learning 
courses,  such  as  web-based  training,  satellite  broadcasts,  and  instructional 
videos.  For  example,  the  services’  acquisition  reform  offices  each  employ 
from  8  to  20  people,  not  nearly  enough  to  provide  on-site  training  or 
in-depth  consulting  to  individual  program  offices.  Some  DOD  organizations 
have  recognized  the  value  of  providing  training  tailored  to  specific  program 
offices  and  are  attempting  to  go  beyond  traditional  training  roles. 

However,  these  are  largely  ad  hoc  efforts  that  do  not  go  as  far  as 
commercial  methods.  For  example,  the  Navy  is  creating  Total  Ownership 
Cost  teams,  that  are  to  advise  program  offices  on  the  use  of  CAIV.  Navy 
officials  stated  that  due  to  limited  resources,  the  teams  will  only  reach 
about  15  of  the  300  program  offices  per  year.  The  Air  Force  plans  to  begin 
using  training  support  teams  to  address  best  practices  issues  from  the 
program  office  perspective.  However,  the  teams  have  not  yet  begun  and 
full  operation  is  not  scheduled  to  begin  until  fiscal  year  2001.  Also,  an  Air 
Force  official  stated  that  another  team,  created  to  address  acquisition 
reform  training  issues,  will  not  involve  program  managers,  which  raises  the 
question  of  whether  program  office  needs  will  be  adequately  considered. 
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The  intensive,  hands-on  training  methods  leading  commercial  firms  employ 
on  new  practices  are  the  result  of  a  systematic,  institutionally  driven 
approach  to  implementation.  These  firms  comnut  to  and  concentrate  their 
resources  and  attention  on  a  few  well-defined  practices  and  make  a 
significant  front-end  investment  in  providing  the  training  to  the  primary 
implementers.  Additionally,  the  firms  strive  to  create  a  supportive 
environment  to  put  the  implementers  of  the  practice  in  a  good  position  to 
succeed.  Their  objective  is  to  have  successful  implementation  at  the  work 
level;  training  methods  are  shaped  to  meet  that  objective.  DOD’s  training 
methods  for  best  practices  do  not  benefit  from  a  similarly  strategic 
approach  for  deploying  new  best  practices  and  provide  little  assurance  that 
the  practices  are  consistently  implemented  in  the  workplace.  DOD  has 
promulgated  numerous  initiatives  in  the  past  few  years  without 
communicating  their  relative  priority  to  trainers  and  implementers.  Often, 
the  initiatives  were  not  accompanied  by  clear  guidance  or  by  the  initial 
training  needed  for  implementation.  While  DOD  commits  significant 
resources  to  training,  it  does  not  make  a  uniform  front-end  investment  to 
ensure  the  implementers  of  new  practices  will  succeed.  This  approach 
depends  more  on  the  individual  program  offices  to  recognize  the  need  for 
and  to  make  this  investment. 

Recent  DOD  actions  reflect  its  recognition  that  training  improvements  are 
needed.  Two  plans  proposing  divergent  improvements  to  the  training  of 
the  acquisition  workforce  are  now  being  considered.  One  plan  does  not 
focus  on  best  practices  training  specifically,  offering  instead  an  incremental 
approach  to  changing  the  DAU  structure.  The  other  targets 
implementation  of  best  practices  directly,  with  a  broad  scope  of  efforts  that 
would  require  substantial  changes  from  the  current  approach  to  training. 
The  latter  plan  is  much  more  aligned  with  the  commercial  approach  to 
training  described  in  this  report.  However,  if  it  is  adopted,  it  faces  many 
challenges  to  providing  better  training  for  program  offices.  Another  recent 
DOD  action — creation  of  a  continuous  learning  policy — may  provide  more 
training,  especially  for  more  senior  members  of  the  acquisition  workforce, 
but  it  does  not  ensure  that  best  practices  will  be  included. 
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Leading  Firms’ 
Strategic  Approach 
Better  Ensures 
Implementation  of  Key 
Practices 


Although  the  four  companies  we  contacted  initiated  very  different  types  of 
practices,  they  exhibited  a  similar  strategic  approach  to  ensuring  that  the 
key  practices  were  implemented.  The  elements  of  this  approach  can  be 
summarized  as 

•  clear  vision, 

•  adoption  of  few  practices  at  any  one  time, 

•  assessment  and  identification  of  target  population  for  implementation, 

•  well-defined  goals, 

•  significant  investment  in  training  and  other  vehicles  to  aid 
implementation,  and 

•  supportive  environment  to  faciUtate  implementation. 


In  general,  we  found  that  the  firms  shared  a  common  strategy  for 
implementing  key  practices  that  were  intended  to  change  company  culture. 
First,  the  firms’  corporate  management  committed  to  and  adopted  seven  or 
less  key  practices  at  any  given  time.  One  firm  we  contacted  adopted  only 
one  or  two  key  new  practices,  which  enabled  the  company  to  concentrate 
its  attention  and  target  resources,  including  training,  to  the  implementation 
of  the  practice,  and  signaled  the  importance  of  the  practices  to  trainers  and 
implementers.  Second,  for  practices  that  were  not  company  wide,  the  firms 
assessed  which  staff  should  be  included  in  the  implementation.  For 
example.  Ford’s  training  unit  assessed  each  existing  program  team  working 
on  vehicle  lines  beyond  the  initial  development  phase  to  determine  if  the 
team  could  benefit  from  adopting  portions  of  FPDS  or  if  some  older  teams 
should  be  excluded.  An  IBM  official  stated  that  while  the  company 
currently  promotes  six  practices,  not  all  will  apply  to  every  segment  of  the 
employee  population.  Third,  once  the  target  population  was  identified, 
company  leaders  made  the  implementation  of  the  practice  mandatory.  For 
example,  the  Ford  Motor  Company  required  that  all  new  vehicle  Unes  built 
for  the  year  2000  and  beyond  use  FPDS.  Lastly,  to  further  assist  in  the 
implementation,  the  companies  developed  well-defined  goals  to  better 
ensure  that  the  target  population  consistently  understood  how  to  apply  a 
new  practice  to  improve  product  production — the  ultimate  goal. 

'The  firms  we  contacted  ensured  that  the  implementers  of  the  practice 
received  the  assistance  necessary  to  succeed.  Consequently,  these 
companies  made  a  significant  investment,  including  a  comprehensive 
front-end  effort,  to  support  the  needs  of  program  offices  that  would 
implement  the  practice.  According  to  a  Ford  training  official,  if  the 
initiative  is  important  enough  to  the  organization,  then  the  investment  is 
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justified.  Other  companies  agreed  with  that  way  of  thinking.  Boeing’s 
Learning  Program  Development  Director  summarized  the  corporate 
training  strategy  for  implementing  new  practices  as  one  that  includes  a 
clearly  stated  vision  or  mission  statement,  well-defined  goals,  and  enablers, 
such  as  training  and  good  processes,  to  support  the  implementers.  This 
philosophy  enabled  Boeing  to  take  a  year  to  develop  the  training  program 
tailored  to  the  777  program — which  was  intended  to  change  the  corporate 
culture  and  encourage  employees  to  rethink  how  they  did  their  jobs.  Both 
Boeing  training  and  program  officials  believe  that  the  training  investment 
resulted  in  the  successful  implementation  of  the  key  777  program 
practices. 

While  the  company  officials  acknowledged  that  training  was  instrumental 
in  the  implementation  of  the  key  practices,  nearly  every  official  we  spoke 
with  stated  that  training  is  just  one  of  the  necessary  components.  They 
noted  that  creating  the  right  environment  is  key  to  the  successful 
implementation  of  new  practices  and  that  the  quality  of  training  was 
dependent  on  this  environment.  Company  officials  stressed  that  strong 
leadership  is  often  another  key.  An  IBM  official  stated  that,  at  inception, 
top  leaders  need  to  provide  sufficient  funding  for  training,  well-defined 
expectations,  clear  direction,  oversight,  continued  interest,  and  incentives 
to  ensure  that  the  new  practices  are  possible  to  implement.  The  manager 
for  the  777  program  stated  that  Boeing’s  management  works  in  teams — a 
key  practice.  He  believed  that  it  was  management’s  ability  to  lead  by 
example  that  helped  to  prevent  a  return  to  the  former  functional  way  of 
operating.  These  companies  believe  that  other  factors,  such  as  an 
accommodating  organizational  structure,  good  internal  communication, 
consistent  appUcation,  and  supportive  technology,  are  needed  to  foster  the 
implementation  of  key  new  practices. 


DOD’s  Training  on  Best 
Practices  Does  Not 
Stem  From  a  Strategic 
Approach 


DOD  does  not  have  a  uniform  or  defined  strategic  approach  to 
implementing  best  practices.  Accordingly,  training  methods  are  not  shaped 
by  the  same  drive  for  implementation  found  in  leading  commercial  firms. 
DOD’s  approach  commonly  begins  with  policy  memorandums  widely 
distributed  to  the  entire  acquisition  workforce.  However,  the  policy 
provides  little  specific  guidance  on  how  or  which  program  offices  should 
implement  the  practice.  Since  1994,  DOD  has  proposed  close  to  40 
acquisition  reform  initiatives  without  an  indication  of  relative  priority.  The 
absence  of  priorities  makes  it  difficult  to  not  only  develop  training  courses 
but  also  to  determine  which  courses  are  most  appropriate  for  the  needs  of 
program  offices. 
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DOD’s  Policy  on  Best 
Practices  Is  Not  Coupled 
With  a  Strategy  for 
Implementation 


DOD’s  policy  on  implementing  the  best  practices  is  promulgated  without  a 
strategy  or  specificity  as  to  whom  within  the  acquisition  workforce  is 
expected  to  implement  the  practices  or  which  program  offices  are  in  the 
best  position,  in  a  program  life  cycle,  to  implement  the  practices.  Typically, 
the  poUcy  is  not  promulgated  in  coiyunction  with  a  detailed  plan  that 
outlines  the  support  and  training  required  by  the  program  offices  tasked 
with  implementing  the  practice.  Similarly,  a  1997  Coopers  &  Lybrand  study 
on  acquisition  reform  implementation  found  inconsistencies  in  DOD’s 
implementation  of  new  practices.  It  stated  that  these  inconsistencies  may 
have  resulted  from  a  lack  of  an  integrated,  cohesive  DOD  strategic  plan  to 
put  them  in  context  to  each  other  and  to  larger  strategic  goals.  While  DOD 
has  made  some  progress  in  establishing  such  goals  through  its  Government 
Performance  Reporting  Act  plans,  organizations  responsible  for  training 
and  implementing  the  practices  have  not  experienced  the  effects. 

In  addition,  the  poUcies  themselves  are  not  always  clear.  For  example, 
although  the  initiative  was  promulgated  in  1995,  training  officials 
reported  they  did  not  have  a  clear  understanding  of  what  CAIV  means. 
AAAV  program  officials  said  they  developed  training  for  1997  to  clarify  the 
CAIV  concept  for  the  program.  A  November  1998  Air  Force  workshop  on 
CAIV  reported  that  the  concept  is  stiU  not  well  understood  or  widely 
implemented. 

Similarly,  the  IPT  policy  states  that  IPTs  should  be  implemented  “when  it 
makes  sense.”  The  poUcy  for  the  use  of  past  performance  data  states  that 
the  practice  should  be  implemented  to  the  “degree  possible.”  While  a  fair 
degree  of  latitude  is  good  for  implementers,  some  DOD  officials  believe 
that  this  level  of  generality  is  insufficient  to  ensure  implementation  of  the 
practice.  For  example,  a  training  official  observed  that  poUcies  are  not  well 
defined  because  of  the  desire  to  bring  new  practices  quickly  to  the 
workforce  and  so  some  are  deployed  when  they  are  not  ready  for 
implementation.  According  to  a  DAU  official,  unclear  and  general  policies 
can  contribute  to  ineffective  training  because  it  makes  it  difficult  to 
develop  a  course  of  instruction  that  teaches  a  unified,  cohesive  way  to 
implement  a  policy.  A  program  official  noted  that  general  policy 
statements  provide  little  guidance  or  assistance  in  the  program  offices’ 
implementation  of  the  practice,  even  though  program  offices  are  often  the 
primary  implementers  of  best  practices. 
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DOD  Attempts  to 
Implement  Many  Practices 
Without  Prioritization 


In  addition  to  IPTs,  performance  specifications,  CAIV,  and  past 
performance,  DOD  has  proposed  numerous  reform  initiatives  since  1994. 
In  1997,  the  Secretary  of  Defense  proposed  several  new  reform  initiatives 
under  the  umbrella  of  the  Defense  Reform  Initiative.  These  initiatives  have 
been  promulgated  without  an  indication  of  relative  priority,  leaving 
educators  and  implementers  to  decide  on  what  is  important.  The  lack  of 
prioritization  makes  it  difficult  to  determine  training  needs,  especially 
when  the  number  of  initiatives  continues  to  increase.  While  the  prolific 
offering  of  new  ideas  can  stimulate  innovation,  the  absence  of  priorities 
makes  it  difficult  to  focus  training  on  the  specific  initiatives  that  are  the 
most  important  to  implement.  Moreover,  program  management  offices  are 
not  necessarily  in  a  good  position  to  sort  through  the  initiatives  to  focus  on 
those  that  are  the  most  important  to  the  job  at  hand. 


Several  DOD  officials  expressed  concerns  regarding  the  number  of 
initiatives  without  prioritization.  For  example,  a  DAU  training  official 
noted  that  it  is  impractical  to  expect  to  train  the  entire  workforce  on  all  of 
the  initiatives  and  that  the  Office  of  the  Secretary  of  Defense  should  set  the 
priorities  for  implementation  of  initiatives.  A  DAU  consultant  echoed  the 
view  that  there  are  too  many  new  reform  initiatives  and  that  DOD  needs  to 
set  priorities  on  which  initiatives  to  address.  A  service  acquisition  reform 
official  observed  that  the  large  numbers  of  best  practices  promoted  by 
DOD  overwhelms  the  services.  This  official  told  us  that  the  three  services’ 
acquisition  reform  offices  meet  quarterly  to  share  progress  on  acquisition 
reform  initiatives.  He  believes  these  meetings  could  serve  as  an 
opportunity  to  prioritize  the  practices.  However,  the  current  focus  of  the 
meetings  is  unclear  and  the  offices’  charters  are  under  review.  Another 
acquisition  reform  office  representative  stated  that  the  combination  of  a 
large  number  of  reform  initiatives  and  unclear  priorities  puts  the  office  in 
the  position  of  having  to  guess  at  what  is  the  most  important.  This  leads  to 
emphasizing  what  is  perceived  to  be  popular,  he  added. 


Oilier  officials  explained  that  due  to  the  frequency  of  new  initiatives  being 
introduced,  training  courses  should  be  regularly  revisited.  However, 
training  personnel  have  not  been  able  to  consistently  keep  up  with  the  pace 
of  change.  For  example,  a  service  acquisition  reform  official  stated  that  it 
requires  at  least  a  full  year  to  determine  the  elements  of  a  new  initiative  and 
then  the  challenge  of  how  to  train  the  initiative  still  exists.  A  1997  DOD 
study  team  that  reviewed  DOD’s  acquisition  education  and  training 
structure  and  process  found  that  curriculum  development  often  lagged 
behind  the  rapidly  changing  policy  requirements.  The  study  also  noted  that 
instructors  were  not  up  to  date  on  current  acquisition  policies  and  that 
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because  they  were  out  of  touch  with  DOD  pohcy  makers,  the  instructors 
found  it  difficult  to  develop  up-to-date  training  courses. 


DOD  is  aware  of  the  need  to  improve  the  means  by  which  the  acquisition 
workforce  receives  and  implements  new  initiatives.  Two  proposals  being 
considered  by  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  that  are  intended  to  improve  workforce  training,  and  a  new 
continuing  education  pohcy  has  been  put  in  place.  The  first  proposal  is  a 
DAU  transition  plan  for  improving  DAU’s  structure.  The  plan  is  based  on  a 
1997  study  by  a  DOD  team  and  a  1998  study  by  the  Logistics  Management 
Institute.  Both  studies  confirmed  strengths  in  the  DAU  approach, 
particularly  relating  to  functional  or  skUl-based  training  and  the 
weaknesses  in  some  of  the  same  areas  that  surfaced  in  our  work.  The 
studies  made  recommendations  for  significant  organizational  and 
operational  changes  in  DAU  to  better  meet  the  educational  needs  of  the 
acquisition  workforce.  However,  a  more  moderate  approach  was  adopted 
in  preparing  the  transition  plan  for  DAU.  While  the  proposed  structure 
offers  improvements,  it  does  not  discemibly  address  key  weaknesses  in  the 
training  of  best  practices.  The  plan  has  yet  to  be  approved.  The  second 
proposal,  a  report  from  the  DOD  Section  912(c)  Commercial  Business 
Environment  Group  directly  advocates  incorporating  best  practices  and  is 
much  broader  in  recommending  changes.  It  proposes  that  DOD’s 
acquisition  structure  be  transformed  into  a  team  type  of  organization  with 
members  drawn  from  across  the  DOD  procurement  enterprise.  A  new 
learning  organization  would  be  created  specifically  to  support  accelerated 
change. 

DOD  has  also  instituted  a  continuous  learning  pohcy,  which  should  help 
people  obtain  current  training,  even  if  they  are  already  fully  certified. 
However,  the  required  levels  of  continuous  learning  can  be  met  in  many 
ways  and  do  not  place  any  particular  emphasis  on  training  in  best 
practices.  These  DOD  efforts  are  a  step  in  the  right  direction  for  seeing 
that  program  offices  have  the  information  they  need,  but  none  focus  on  the 
program  offices’  needs.  Given  the  pivotal  role  they  play  in  weapons 
acquisition,  a  more  direct  approach  may  be  required. 


DOD  Proposals  to  Alter 
Training  Reflect 
Conflicting 
Approaches 
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DAU  Transition  Plan  Is 
Largely  Silent  on 
Improvements  to  Best 
Practices  Training 


The  DOD  transition  plan  for  restructuring  DAU  was  based  on 
recommendations  of  a  1997  process  action  team  that  DOD  chartered  to 
create  a  clear  vision  and  a  structure  for  future  workforce  training  and 
education.  Several  findings  of  the  study  related  to  training  for  new 
initiatives  and  practices,  including  (1)  the  existing  curriculum  design 
process  was  functionally  driven,  did  not  address  the  needs  of  people 
outside  the  functional  career  fields,  and  did  not  lend  itself  to  successful 
development  of  cross-functional  curricula;  (2)  a  mechanism  was  needed  to 
validate  and  prioritize  requirements  before  they  are  submitted;  (3)  faculty 
members  did  not  incorporate  up-to-date  case  studies;  (4)  DAU  schools 
used  procedures  for  measuring  performance  that  focused  on  student 
satisfaction  with  a  course,  not  whether  the  user  community’s  acquisition 
education  requirements  were  being  met;  and  (5)  DAU’s  use  of  technology- 
based  learning  was  insufficient. 

The  team  recommended  that  DAU  be  replaced  with  a  unified  Defense 
Acquisition  Institute  that  would  have  responsibility  for  development  and 
delivery  of  acquisition  training,  the  consortium  be  streamlined  to  reduce 
duphcation,  and  lines  of  authority  between  DAU  and  the  members  be 
strengthened.  The  team  envisioned  the  institute  as,  among  other  things,  (1) 
fostering  iimovation  and  facilitating  reform  and  continuous  improvements, 
(2)  having  a  clear  message  of  support  from  DOD  acquisition  leadership  that 
education  is  the  key  to  meeting  DOD  goals,  (3)  identifying  acquisition 
training  needs  that  are  cross-functional  areas  or  that  are  multifunctional  in 
nature,  (4)  focusing  on  the  needs  of  the  learner,  and  (5)  emplosdng  a 
process  to  validate  the  effectiveness  of  the  training  received.  However,  a 
dissenting  opinion  offered  by  one  of  the  team  members  to  evolve  the 
current  DAU  organization  structure  through  a  series  of  near-term  and 
long-term  actions  was  accepted  by  the  Under  Secretary  of  Defense. 

Subsequently,  the  Logistic  Management  Institute  conducted  a  review  for 
DOD  and  developed  a  structural  reorganization  for  DAU  that  was  to  be 
used  as  the  starting  point  for  the  transition  plan.  The  Institute  noted  that 
DAU  training  was  generally  skill  and  specialty  oriented,  with  limited 
overlap  among  “stove-piped”  career  fields.  It  recommended  that  the  new 
structure  be  designed  along  the  lines  of  a  corporate  university.  As  such,  its 
curriculum  would  include  cross-functional  training  and  business  executive 
development  and  act  as  an  agent  of  change,  being  more  involved  with  the 
workforce  in  determining  how  changes  would  be  carried  out. 

The  DAU  president  submitted  the  draft  transition  plan  for  restructuring  in 
September  1998.  As  of  June  1999,  DOD  had  not  approved  the  plan.  The 
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plan  has  several  features  that  offer  promise  for  more  effective  training 
overall.  These  include  (1)  a  unified  structure  in  which  DAU  has  full 
responsibility  for  ensuring  that  the  overall  course  structure  meets  the 
needs  of  the  acquisition  workforce,  (2)  consolidation  of  the  consortium 
schools  into  four  main  campuses  under  the  direct  authority  of  DAU,  (3)  an 
increased  emphasis  on  distance  learning  techniques,  (4)  steps  to  improve 
the  qualifications  of  faculty,  and  (5)  replacement  of  functional  boards  with 
functional  IPTs  that  have  representation  fi-om  both  functional  experts  and 
instructors  and  that  will  design  curricula  together. 

Other  features  of  the  plan  raise  doubts  about  its  ability  to  provide  a 
strategic  approach  to  shape  training  that  will  help  implementation  of  best 
practices  at  program  offices.  The  transition  team  that  prepared  the  plan 
concluded  that  DAU  was  unique  and  could  not  be  matched  with  other 
institutions,  like  corporate  universities.  Thus,  the  corporate  university 
model  recommended  by  the  Institute  was  not  adopted.  While  the 
curriculum  may  benefit  fi'om  the  functional  IPTs,  the  plan  does  not  mention 
the  need  for  cross-functional  or  best  practices  training  or  for  DAU  to 
accept  the  role  of  a  change  agent.  It  is  silent  on  prioritizing  initiatives  or 
improving  feedback.  While  it  is  possible  that  under  the  proposed  plan  DAU 
could  play  a  more  active  role  in  the  design  and  delivery  of  tailored  training 
to  the  workplace,  the  plan  does  not  that  suggest  that,  other  than  increased 
distance  learning,  training  will  be  offered  in  a  manner  substantially 
different  from  what  has  been  traditionally  offered  or  that  the  relationship 
between  DAU  and  the  acquisition  community  will  be  any  closer. 


Commercial  Business 
Environment  Plan  Proposes 
Major  Changes  to  Hasten 
Reform 


The  National  Defense  Authorization  Act  for  Fiscal  Year  1998,  section  912, 
directed  the  Secretary  of  Defense  to  submit  to  Congress  an  implementation 
plan  to  streamline  the  acquisition  organizations,  workforce,  and 
mfrastructure.  As  part  of  that  mandate,  the  Deputy  Under  Secretary  of 
Defense  for  Acquisition  and  Technology  formed  a  team  to  develop  a  plan 
for  ensuring  that  new  practices  would  be  incorporated  by  DOD.  The  draft 
plan,  dated  June  1999,  concludes  that  adopting  the  most  effective 
commercial  practices  requires  a  cultural  and  organizational  transformation 
within  DOD.  The  draft  proposes  a  very  broad  approach  to  making  this 
change,  which  includes  a  training  regime  to  implement  key  commercial 
business  practices  and  a  change  model  to  accelerate  the  implementation  of 
new  practices.  The  model  calls  for  establishing  goals  and  a  scorecard  to 
measure  whether  they  are  met.  To  inform  everyone  of  the  goals,  a  tiered 
approach  is  envisioned,  starting  with  senior  leaders,  rolling  down  to  teams 
with  specific  reform  target  goals.  The  teams  are  to  report  on  their 
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outcomes  within  60  days.  A  mission  support  office  will  support  the  teams 
as  well  as  capture  results  across  DOD  and  make  them  available  to  others. 
This  office  is  to  be  led  by  a  senior  DOD  official  and  staffed  by  management 
and  facilitation  experts,  and  possibly  outside  contractors. 

The  plan  proposes  a  team  approach  for  managing  acquisitions  that 
embraces  best  practices  and  maintains  “learning  organizations  that  seek 
out  and  adopt  best  practices  that  improve  individual  and  organizational 
performance.”  The  plan  also  calls  for  new  organizational  roles  in  adopting 
of  best  practices.  Among  these  is  DAU.  The  plan  recommends  that  DAU  be 
broadly  recast  to  adopt  the  corporate  university  model  and  become  a 
change  agent.  This  plan,  while  not  specific  about  the  help  that  program 
offices  would  receive,  calls  for  a  strategic  approach  that  would  make  it 
more  likely  that  DOD  could  provide  its  program  offices  tailored  training- 
more  help — in  implementing  best  practices.  Nonetheless,  translating  this 
plan  into  the  type  of  training  and  other  help  program  offices  need  to 
implement  best  practices  is  challenging.  Key  elements,  such  as  the 
cascading  of  goals  from  senior  levels  on  down  and  the  revision  of  what 
constitutes  teams,  are  migor  undertakings  and  challenge  long  established 
patterns  of  interaction.  How  the  mission  support  office  would  reach 
specific  program  offices  is  unclear,  as  is  how  it  would  interface  with  DAU. 
Also,  whether  initiatives  are  to  be  prioritized  and  how  DAU  would  be 
restructured  to  provide  functional  training  and  become  a  change  agent  for 
new  practices  remains  to  be  delineated. 


Continuous  Learning  Policy 
Does  Not  Ensure  Best 
Practices  Training 


The  objective  of  DOD’s  recently  issued  continuous  learning  policy  is  to 
augment  the  acquisition  training  standards  for  career  field  certification  of 
acquisition  professionals.  The  policy  requires  all  civilian  and  military 
members  of  the  acquisition  workforce  to  complete  80  hours  of  continued 
education  within  a  2-year  period.  As  such,  it  is  a  good  step  for  reaching 
more  senior  people  in  the  acquisition  workforce  who  would  normally  not 
receive  this  kind  of  training  because  they  are  fiilly  certified.  The  policy  also 
places  increased  emphasis  on  distance  learning,  which  may  address  some 
of  the  timeliness  and  availability  issues  raised  by  program  office  officials  in 
obtaining  useful  best  practice  information.  One  distance  learning  course 
that  deals  directly  with  a  best  practice  was  jointly  developed  last  year  by 
the  National  Contract  Management  Association  and  the  National 
Association  of  Purchasing  Management  at  the  request  of  DOD.  It  is  an 
internet-based  course  to  integrate  best  commercial  practices  for  managing 
suppliers  into  DOD  acquisitions.  It  entitles  participants  to  24  hours  of 
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continuous  education  credit  and  participants  are  encouraged  to  take  the 
course  as  teams. 

While  the  policy  does  not  preclude  staff  from  receiving  training  on  new 
practices,  given  the  array  of  options  for  meeting  requirements,  it  does  little 
to  ensure  that  best  practices  training  will  be  received.  An  earlier  version  of 
the  policy  recommended  16  units  of  acquisition  reform  training  that  could 
have  included  best  practices,  but  the  requirement  was  deleted.  The  policy 
encourages  acquisition  reform  training  but  allows  for  requirements  to  be 
met  in  many  ways — ^including  courses,  professional  activities,  and  on  the 
job  experience — which  may  or  may  not  include  best  practices  information. 
According  to  a  training  official,  crediting  acquisition  workforce  members 
for  almost  anything  dilutes  the  policy. 

Officials  of  one  program  noted  that  no  one  advises  them  on  what  courses  to 
take.  They  can  take  courses  or  attend  conferences  that  they  believe  would 
benefit  them,  but  they  believed  this  may  be  inadequate  for  ensuring  that 
senior  officials  are  aware  of  new  ideas.  Another  program  official  said  that 
her  program  office  could  meet  continuing  education  requirements  by 
mandatory  attendance  of  Acquisition  Reform  Week.  While  this  exposure 
would  increase  awareness  of  new  practices,  such  training  was 
characterized  as  providing  just  a  “flavor”  for  the  initiative — not  providing 
enough  information  to  implement  the  practice. 
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Leading  commercial  firms  and  weapon  system  program  offices  have 
successfully  implemented  best  practices  or  new  initiatives  by  taking  a 
tailored,  on-site  training  approach.  Such  an  approach  is  a  concentrated, 
up-front  effort  that  creates  an  environment  for  accepting  the  practice 
within  a  program  and  provides  the  impetus  and  know-how  to  apply  the 
practice  at  the  workplace.  It  is  warranted  by  the  investment  and 
importance  that  large  programs  represent.  The  difference  is  that  within 
DOD,  the  approach  is  allowed  to  happen,  while  leading  commercial  firms 
ensure  it  happens.  These  firms  ensure  that  initiatives  are  successfully 
implemented  at  the  program  level  by  customizing  the  initiatives  to  fit  the 
programs,  working  with  program  office  staff  to  develop  training  that  meets 
the  needs  of  the  program,  and  providing  on-site  assistance  to  the  program. 
This  approach  would  not  necessarily  be  used  every  time  by  conunercial 
firms  to  train  their  entire  workforce  or  to  impart  every  skill;  it  is  instead  an 
intensive  approach  they  reserve  for  key  initiatives  and  individual  programs. 
It  stems  from  an  overall  training  strategy  that  restricts  the  number  of 
initiatives  that  are  brought  to  the  workforce,  ensures  that  the  initiatives 
have  well-defined  objectives,  and  makes  training  organizations  responsible 
for  seeing  that  policy  is  turned  into  practice  at  the  program  level.  Little 
regarding  implementation  is  left  to  chance. 

Within  DOD,  this  approach  can  be  taken  if  a  program’s  management  has  the 
insight  to  recognize  the  importance  of  an  initiative,  the  necessary  resources 
needed  to  implement  the  initiative  at  the  work  site,  and  support  from  top 
management.  Even  though  DOD  has  draAvn  its  reform  initiatives  fi-om  the 
practices  of  leading  commercial  firms,  it  has  not  adopted  the 
accompanying  training  strategy  to  ensure  a  concerted  effort  is  made  to 
implement  key  initiatives  at  the  program  level.  As  a  result,  more  is  left  to 
chance.  In  the  past  4  years,  over  40  initiatives  have  been  introduced  to  the 
DOD  acquisition  workforce,  without  delineating  which  are  the  most 
important  for  weapon  system  programs.  Several  organizations  are 
responsible  for  training  and  each  service  has  an  office  that  promotes 
acquisition  reform  initiatives.  Although  these  organizations  are  doing 
valuable  work  and  are  contributing  to  workforce  training,  it  is  unclear  how, 
collectively,  they  can  drive  key  initiatives  to  the  program  office  level. 

DAU’s  main  tool  for  training  the  acquisition  workforce — afunctional 
training — is  aimed  at  increasing  the  expertise  of  people  in  their  career 
fields,  but  it  does  not  have  the  throughput  and  ability  to  reach  the  program 
office  as  a  work  unit,  nor  the  practical  depth  needed  to  imbed  an  initiative 
at  the  program  office.  Other  tools,  such  as  satellite  broadcasts,  distance 
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learning,  and  Acquisition  Reform  Week,  make  program  office  staff  aware  of 
new  initiatives,  but  they  do  not  provide  the  depth  or  environment  to 
implement  them  because  DOD’s  training  organizations  and  methods  have 
not  been  designed  to  provide  this  kind  of  support. 

DOD  recognizes  the  importance  of  training  as  a  tool  that  can  help  infuse 
best  practices  into  weapon  acquisitions  and  has  several  draft  plans  to 
reorganize  how  training  is  delivered.  However,  it  is  not  evident  that  the 
planning  and  methods  called  for  by  the  DAU  transition  plan  will  change  the 
way  such  training  is  conceived  and  delivered  to  weapon  system  program 
offices.  The  Commercial  Business  Environment  Plan  does  call  for  a 
strategic  approach  that  would  make  it  more  likely  that  DOD  could  provide 
its  program  offices  tailored  training — ^more  help — ^in  implementing  best 
practices.  The  fact  that  neither  plan  has  been  approved  indicates  that  DOD 
has  not  decided  what  role  it  wants  acquisition  training  to  play  on  best 
practices. 

Whether  through  existing  organizations  or  through  new  ones,  DOD  needs 
to  increase  its  capability  to  provide  tailored  training  of  specific  initiatives  at 
the  program  office  level.  Developing  this  additional  capability  will  require 
a  strategy  for  implementation  and  collaboration  with  program  offices  on 
the  design  of  training.  Without  a  concerted  approach  to  foster  the 
implementation  of  best  practices  by  providing  for  customized  training  at 
the  program  offices,  better  outcomes  in  weapon  programs  will  be  more 
difficult  to  achieve.  The  recommendations  that  follow  address  ways  DOD 
can  provide  tailored  training  on  selected  practices  to  program  offices.  The 
recommendations  are  made  without  prejudice  toward  the  functional  and 
other  training  DOD  provides  to  the  acquisition  workforce. 


We  recommend  that  the  Secretary  of  Defense  develop  a  strategy  for 
delivering  targeted  training  on  selected  new  practices  to  program  offices  to 
ensure  the  practices  will  be  implemented.  This  strategy  should 
accomplish  the  following. 

•  Identify  those  initiatives  most  worthy  of  a  targeted  training  investment 
by  screening  the  numerous  initiatives  to  separate  those  for  which  a 
general  awareness  is  sufficient  from  those  that  warrant  a  targeted 
approach.  Those  to  be  targeted  should  be  relatively  few  in  number  at 
any  given  point  in  time.  The  process  of  setting  priorities  would  be  of  a 
continuing  nature  and  would  benefit  from  the  input  of  current  program 
office  members. 
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•  Decide  whether  DAU  is  to  play  a  more  traditional  role,  as  called  for  by 
the  DAU  Transition  Plan,  or  a  more  proactive  role,  as  called  for  by  the 
Commercial  Business  Environment  Plan  and  clearly  communicate  to 
DAU  and  other  providers  of  DOD  training  their  responsibilities  in 
supporting  a  targeted  approach  to  training. 

•  Identify  the  key  organizations  and  people  that  are  critical  to  the  program 
offices’  ability  to  implement  best  practices,  including  those  not  currently 
defined  as  part  of  the  acquisition  workforce  and  contractors,  as  the 
potential  audience  of  targeted  training. 

We  also  recommend  that  the  Secretary  make  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology  responsible  for  taking  steps  to 
institutionalize  the  methods  for  tailoring  training  on  key  initiatives  that 
have  been  shown  to  be  successful  by  leading  program  offices  and 
commercial  firms.  This  involves  having  proactive  organizations  and  the 
tools  to  inform,  prepare,  and  assist  program  offices  to  implement  the 
initiatives  most  applicable  to  their  programs.  This  approach  should  include 
the  following. 

•  Making  new  program  managers  aware  of  the  initiatives  that  could 
significantly  affect  the  outcomes  of  their  programs  and  the  role  the 
managers  play  in  having  their  staffs  trained  to  implement  those 
initiatives.  This  could  be  done  through  existing  program  managers’ 
courses. 

•  Working  with  individual  program  managers  to  tailor  the  initiatives  to 
their  programs. 

•  Developing  an  approach,  in  conjunction  with  the  program  managers,  to 
create  a  culture — ^and  necessary  incentives — within  the  program  office 
to  make  it  receptive  for  adopting  the  initiatives. 

•  Helping  the  program  managers  determine  the  best  methods  for  making 
initial  and  sustaining  resource  investments  in  training  the  program 
office  staff.  This  help  could  consist  of  providing  advice,  assisting  in  the 
identification  of  experts  in  the  needed  areas,  and  culling  lessons  learned 
from  other  programs  that  have  implemented  the  initiative. 

To  introduce  and  reinforce  new  practices,  we  recommend  that  the 
Secretary  require  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  more  effectively  use  existing  training  vehicles  to 

•  incorporate  new  practices  more  quickly, 

•  better  reflect  the  changing  role  of  program  managers. 
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•  ensure  that  instructors’  knowledge  keeps  pace  with  the  latest  practices, 
and 

•  provide  more  case  study  material  that  gives  current  implementation 
examples  for  a  variety  of  situations. 


Agency  Comments  and 
Our  Evaluation 


DOD  concurred  with  a  draft  of  this  report  and  all  of  its  recommendations 
(see  app.  I).  DOD  noted  that  it  was  taking  steps  to  develop  a  strategy  for 
delivering  targeted  training  on  selected  new  practices  to  program  offices, 
consistent  with  our  report.  It  cited  the  Commercial  Business  Environment 
study  team’s  vision  for  accelerating  cultural  change  within  the  acquisition 
community,  in  coryunction  with  a  strategy  of  delivering  team  training,  to 
implement  best  practices.  DOD  stated  that  it  anticipates  adopting  and 
launching  many  of  the  study’s  recommendations  without  delay.  DOD  also 
recognized  that  DAU  should  augment  its  capability,  consistent  with  the  best 
practices  of  the  private  sector  corporate  universities,  by  embracing  the  role 
of  “change  agent”  and  by  designating  “performance  consultants”  who  focus 
on  developing  tailored  training  to  meet  program  team  needs. 


DOD  noted  that  it  has  also  begun  the  restructuring  of  DAU  to  provide  a 
more  centralized  and  integrated  education  and  training  program  that  will 
also  be  integrated  with  the  initiatives  identified  by  the  Commercial 
Business  Environment  report.  We  agree  that  the  DOD  study  frames  a 
strategic  approach  that  would  make  it  more  likely  that  DOD  could  provide 
its  weapon  system  program  offices  tailored  training  to  help  them 
implement  new  practices.  We  are  also  encouraged  by  the  speed  with  which 
DOD  is  implementing  the  study’s  recommendations.  We  do  note  that  the 
study  is  not  specific  about  the  help  that  program  offices  would  receive  and 
that  translating  the  study  into  such  specifics  will  be  a  significant  challenge. 
As  DOD  proceeds  with  implementation,  we  reiterate  the  importance  of 
screening  the  numerous  initiatives  to  identify  those  most  worthy  of 
targeted  investments  and  to  involve  the  staff  of  current  program  offices  in 
setting  these  priorities. 


In  agreeing  on  the  need  for  targeted  training,  DOD  noted  that  DAU’s 
communications  arm  will  work  to  provide  for  immediate  deUveiy  of 
tailored  training  on  key  initiatives  to  meet  the  needs  of  program  offices. 
DAU’s  current  communications  arm — ^the  ARCC — ^provides  training 
primcuily  by  distributing  videos  and  periodic  satellite  broadcasts.  As  such, 
ARCC  may  not  be  well-poised  to  take  on  the  responsibilities  of  a  proactive 
organization  with  the  tools  needed  to  inform,  prepare,  and  assist  program 
offices  to  help  them  implement  the  initiatives  most  applicable  to  their 
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programs,  as  we  have  recommended.  Regardless  of  which  organization  is 
assigned  the  responsibility,  it  is  important  that  the  organization  go  beyond 
the  traditional  approach  of  making  standard  training  available  and  instead 
work  with  program  offices  to  (1)  tailor  the  initiatives  to  their  programs; 

(2)  develop  an  approach  to  help  make  their  staffs  receptive  to  adopting 
new  practices;  and  (3)  help  the  program  managers— on  site,  if  necessary— 
determine  the  best  methods  for  making  the  investments  in  training  the 
program  office  staff. 

Regarding  making  more  effective  use  of  existing  training  vehicles,  DOD 
stated  that  during  a  review  of  core  curriculum  requirements,  it  will  be 
completely  reviewing  the  principal  program  management  training  toward 
implementing  a  fully  integrated  strategy.  It  noted  that  the  strategic  plan 
will  address  the  development  of  tailored  case  study  materials,  practical 
exercises,  and  assessment  criteria  for  the  adoption  of  new  practices.  We 
believe  that  DOD’s  proposed  review  has  the  potential  to  more  effectively 
promote  best  practices  training.  In  addition  to  these  actions,  as  DOD 
conducts  its  review,  it  needs  to  have  existing  training  vehicles  incorporate 
new  practices  more  quickly,  better  reflect  the  changing  role  of  program 
managers,  and  ensure  that  instructors’  knowledge  keeps  pace  with  the 
latest  practices.  Greater  involvement  of  weapon  system  program  office 
staff  in  the  design  and  content  of  these  training  vehicles  could  link  the 
training  more  closely  with  the  job  at  hand.  While  DOD  plans  to  focus  its 
continuous  learning  activities  on  key  acquisition  reforms,  currently, 
members  of  the  acquisition  workforce  do  not  necessarily  have  to  take 
training  on  acquisition  reform  to  meet  continuous  learning  requirements. 
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THE  UNDER  SECRETARY  OF  DEFENSE 

3010  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3010 


ACQUtSITION  AND 
TECHNOLOGY 


JUI.  2  ^  1999 


Mr.  Louis  J.  Rodrigues 
Director,  Defense  Acquisition  Issues 
National  Security  and  International 
Affairs  Division 

U.S.  General  Accounting  Office 
Washington,  D.C.  20548 


Dear  Mr.  Rodriques: 


This  is  the  Department  of  Defense  (DOD)  response  to  the  General  Accounting 
Office  (GAO)  draft  report,  “BEST  PRACTICES:  DoD  Training  Can  Do  More  to  Help 
Weapon  System  Programs  Implement  Best  Practices”  (GAO/NSI AD-99-206). 

The  Department  concurs  with  the  GAO  draft  report  and  following  the  path  of 
leading  commercial  firms,  the  Department  Is  already  implementing  many  of  the  best 
practice  recommendations  and  considering  others  in  the  Commercial  Business 
Environment  Team  Report  on  “Accelerating  Change  Through  Enterprise  Teaming.”  The 
Department  welcomes  the  GAO’s  draft  report  and  specific  responses  to  the 
recommendations  are  included  herein  as  an  attachment.  Suggested  technical 
comments  and  changes  for  clarification  and  accuracy  have  been  provided  separately. 

Thank  you  for  the  opportunity  to  review  and  comment  on  the  report.  The  level  of 
cooperation  between  your  staff  and  mine  Is  greatly  appreciated  and  we  look  fonvard  to 
working  with  your  staff  again  in  the  future. 

Sincerely, 


J*  S.  Gan^r 


Attachment 
As  stated 
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GAO  Draft  Report 

“Best  Practices:  DOD  Training  Can  Do  More  to  Help  Weapon  System  Programs 
Implement  Best  Practices” 

(GAO/NSIAD-99-206) 

Department  of  Defense  Comments 
***** 


GAO  Recommendation  1 :  The  GAO  recommends  that  the  Secretary  of  Defense  develop 
a  strategy  for  delivering  targeted  training  on  selected  new  practices  to  program  offices 
to  ensure  the  practices  will  be  implemented. 

DOD  Response:  Concur  with  comment.  While  the  Department  agrees  with  the 
recommendation  in  general,  it  is  important  to  note  the  steps  already  being  undertaken 
that  are  consistent  with  the  GAO  Report.  The  Department  of  Defense’s  Commercial 
Business  Environment  implementation  study  team,  established  under  Section  912  of  the 
National  Defense  Authorization  Act  for  Fiscal  Year  1998,  developed  a  vision  for 
accelerating  cultural  change  within  the  Defense  acquisition  community.  The  report  is 
currently  in  final  coordination  and  the  Department  anticipates  adopting  and  launching 
many  of  Its  recommendations  without  delay.  Indeed  the  change  acceleration  model  is 
already  being  demonstrated  in  both  the  policy  and  business  practice  arenas.  This  vision 
is  focused  on  an  action  team  model  that  would  ensure  that  the  Defense  acquisition 
community  through  change  acceleration  management  in  conjunction  with  a  strategy  of 
delivering  team  training  implements  best  practices.  With  regard  to  the  latter,  we  have 
piloted  several  initiatives  by  (1)  cooperatively  developing  an  initial  interactive,  on-line 
team  training  course  on  “Supplier  Managemenf  with  the  National  Association  of 
Purchasing  Management  and  the  National  Contract  Management  Association;  and  (2) 
customizing  an  Executive  Education  Course  at  the  University  of  Virginia’s  Darden 
Graduate  School  of  Business  on  “Competing  in  a  New  Business  Environmenf,  under 
the  auspices  of  DAU  and  the  U.  S.  Army.  The  Department  recognizes  that  DAU  should 
augment  its  capability,  consistent  with  the  best  practices  of  the  private  sector  corporate 
universities,  by  embracing  the  role  of  “change  agenf  and  by  designating  “performance 
consultants”  who  focus  on  developing  tailored  training  to  meet  program  team  needs. 

DoD  has  also  begun  the  restructuring  of  DAU  to  provide  a  more  centralized  and 
integrated  education  and  training  program  that  will  also  be  integrated  with  the  initiatives 
identified  by  the  Commercial  Business  Environment  Report  and  targeted  to  our  new 
business  practices. 

GAO  Recommendation  2:  The  GAO  recommends  that  the  Secretary  of  Defense  make 
the  Under  Secretary  of  Defense  for  Acquisition  and  Technology  responsible  for  taking 
steps  to  institutionalize  the  methods  for  tailoring  training  on  key  initiatives  shown  to  be 
successful  by  leading  program  offices  and  commercial  firms.  This  involves  having 
proactive  organizations  and  the  tools  needed  to  inform,  prepare,  and  assist  program 
offices  to  help  them  Implement  the  initiatives  most  applicable  to  their  programs. 

POP  Response:  Concur.  Although  existing  DAU  program  management  training  make 
new  managers  aware  of  initiatives  that  could  significantly  affect  the  outcomes  of  their 
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programs  and  the  role  managers  play  In  getting  their  staffs  trained  to  implement  those 
initiatives,  even  more  emphasis  is  needed  to  ensure  new  initiatives  targeted  training  and 
are  incorporated  Into  existing  training  where  appropriate.  The  communications  arm  of 
the  DAU  will  work  to  provide  for  immediate  delivery,  through  a  variety  of  means,  of  the 
key  initiatives  shown  to  be  successful  by  leading  program  offices  and  commercial  firms, 
yet  tailored  to  be  most  applicable  to  the  program  office  in  need  of  the  training. 

GAO  Recommendation  3:  The  GAO  recommends  that  the  Secretary  of  Defense  require 
the  Under  Secretary  of  Defense  for  Acquisition  and  Technology  to  more  effectively  use 
existing  training  vehicles  to  introduce  and  reinforce  new  practices. 

POD  Response:  Concur.  During  a  review  of  core  curriculum  requirements  as  part  of  the 
new  continuous  learning  policy,  the  Department  has  started  a  review  of  the  principal 
program  management  training  (Including  on/off-sIte  training  modules  and  alternative 
commercial  training  offerings)  toward  implementing  a  fully  integrated  strategy.  The 
overall  strategic  plan  will  address  the  development  of  tailored  case  study  materials, 
practical  exercises,  and  assessment  criteria  for  arriving  at  acceptable  solutions 
premised  on  the  adoption  of  new  practices.  As  well,  this  effort  seeks  to  focus  the 
Department’s  continuous  learning  activities  on  key  acquisition  reforms,  including  the 
creation  of  specific,  widely  accessible  learning  modules. 


Page  66 


GAO/NSIAD-99-206  Best  Practices 


Appendix  II _ _ _ 

Description  of  Program  Offices  Visited 


Following  is  a  description  of  each  of  the  program  offices  we  visited. 


Advanced  Amphibious 
Assault  Vehicle 


The  Advanced  Amphibious  Assault  Vehicle  (AAAV)  is  to  be  used  by  the 
Marine  Corps  to  provide  high-speed  transportation  of  troops  from  ships 
located  beyond  the  horizon  to  the  beaches.  It  will  also  provide  armor 
protection,  land  mobility,  and  fire  support  during  combat  operations  on  the 
shore.  It  is  a  category  F  major  acquisition  program  for  the  Marine  Corps. 
The  program  office  is  staffed  by  about  75  people. 


Advanced 
Medium-Range 
Air-to-Air  Missile 


The  Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM)  is  a  new 
generation  radar-guided  missile  that  fighter  aircraft  are  to  use  against 
enemy  aircraft.  It  can  be  launched  beyond  visual  range,  day  or  night  and  in 
all  weather.  The  missile  is  used  on  the  Air  Force  F-15  and  F-16  and  the 
Navy  F-14  and  F/A-18.  It  is  a  category  I  acquisition  program  for  the  Air 
Force.  The  AMRAAM  program  started  over  20  years  ago.  About  70  people 
staff  the  program  office. 


Battlefield  Combat 
Identification  System 


The  Battlefield  Combat  Identification  System  (BCIS)  is  a  millimeter  wave 
electronic,  question,  and  answer  combat  identification  system  capable  of 
identifying  friendly  ground  combat  vehicles.  The  BCIS  interrogation  is 
triggered  automatically  by  activation  of  the  shooter,  which  sends  an 
encrypted  query  message  to  the  targeted  vehicle.  If  the  targeted  vehicle  is 
friendly  and  equipped  with  BCIS,  its  transponder  answers  with  an 
encrypted  friend  message  that  is  illvuninated  in  the  shooter’s  sights.  It  is  a 
category  II^  acquisition  managed  by  the  Army.  The  program  office  is 
staffed  by  16  core  people  and  30  additional  people  who  are  assigned  from 
other  government  organizations  or  contractor  support. 


‘  Category  I  programs  are  defined  as  major  defense  acquisition  programs  estimated  to  cost  over 
$355  million  for  research,  development,  test,  and  evaluation,  or  have  procurement  costs  of  more  than 
$2,135  billion. 

^  Category  II  programs  are  defined  as  acquisition  programs  estimated  to  cost  over  $75  million  for 
research,  development,  test  and  evaluation,  or  have  procurement  costs  of  more  than  $300  million. 
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Appendix  II 

Description  of  Program  Offices  Visited 


Joint  Air-to-Surface 
Standoff  Missile 

The  Joint  Air-to-Surface  Standoff  Missile  (JASSM)  is  an  autonomous, 
long-range  cruise  missile  to  be  capable  of  launch  from  outside  area 
defenses  to  hit  ground  targets.  It  is  to  be  launched  from  a  wide  range  of 
bomber,  attack,  and  fighter  fixed-wing  aircraft.  It  is  a  joint  Air  Force  and 
Navy  category  I  mjgor  acquisition  program. 

Joint  Strike  Fighter 

The  Joint  Strike  Fighter  (JSF)  program’s  objective  is  to  develop  and  field  an 
affordable,  highly  common  family  of  next  generation  multirole  strike 
fighter  aircraft  for  the  Navy,  the  Air  Force,  the  Marine  Corps,  and  U.S. 
allies.  The  focus  of  the  program  is  affordability.  It  is  a  joint  service 
category  I  major  acquisition  program  with  Navy  as  lead.  About  100  people 
staff  the  program  office. 

Joint  Surveillance  and 
Target  Attack  Radar 
System-Joint  Tactical 
Terminal 


Terminal  (JSTAES-JTT)  is  to  provide  warfighters  with  near  real-time 
tactical  intelligence  and  targeting  information.  It  is  a  terminal  that  supplies 
the  critical  data  link  from  various  intelligence  sources  to  battle  managers 
across  all  services.  The  terminal  is  integrated  into  other  weapon  systems 
and  is  to  be  mounted  in  fixed-  and  rotary-wing  aircraft,  surface  ships,  and 
fixed  or  mobUe  ground  platforms  and  vehicles.  It  is  managed  by  the  Army 
and  is  a  category  III^  program.  About  10  people  staff  the  program  office. 


®  Category  III  programs  are  defined  as  those  acquisitions  programs  that  do  not  meet  the  criteria  for 
categories  I  or  11. 
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